Learn the best practices and efficient techniques to manipulate and reshape data using pandas.

Machine Learning Essentials

This module is a comprehensive guide to using pandas for data analysis. It covers a wide range of topics related to data manipulation, including filtering, merging, grouping, pivoting, and reshaping data. In this module, we’ll learn about the best practices and efficient techniques for working with data. We’ll encounter numerous examples of real-world data analysis problems and will gain hands-on experience with the pandas library to solve them effectively. This module will also cover performance considerations and provide insights on common pitfalls to avoid when using pandas. The module is suitable for both beginners and experienced users of pandas. It includes detailed explanations of pandas concepts and functions, as well as tips and tricks for optimizing our code. At the end of this module, we’ll be able to effectively handle data using pandas.

Effective Data Manipulation with pandas

Various patterns pop up when we start analyzing data, giving us the tools that we need to deal with them.

## Dummy columns
Creating dummy columns is one way to convert a categorical column into numeric columns. The process is straightforward. If we have a column that has repeated string values, we can create a new column for each of those values and insert a 1 or a 0 in each new column if it corresponds to the original value.

We'll look at a concrete example using the JetBrains Python 2020 survey data. The job columns are almost in dummy format as is. But instead of having entries of 1 and 0, they have entries of the job title and `NaN`:


Various patterns pop up when we start analyzing data, giving us the tools that we need to deal with them.

# Dummy columns
Creating dummy columns is one way to convert a categorical column into numeric columns. The process is straightforward. If we have a column that has repeated string values, we can create a new column for each of those values and insert a 1 or a 0 in each new column if it corresponds to the original value.

We'll look at a concrete example using the JetBrains Python 2020 survey data. The job columns are almost in dummy format as is. But instead of having entries of 1 and 0, they have entries of the job title and `NaN`:


Explore various options for manipulating and reshaping a DataFrame.

Introduction

Series Deep Dive

DataFrames

Manipulating Data

Conclusion

Reshaping DataFrames with Dummies

Dummy columns