Learn the best practices and efficient techniques to manipulate and reshape data using pandas.

Machine Learning Essentials

This module is a comprehensive guide to using pandas for data analysis. It covers a wide range of topics related to data manipulation, including filtering, merging, grouping, pivoting, and reshaping data. In this module, we’ll learn about the best practices and efficient techniques for working with data. We’ll encounter numerous examples of real-world data analysis problems and will gain hands-on experience with the pandas library to solve them effectively. This module will also cover performance considerations and provide insights on common pitfalls to avoid when using pandas. The module is suitable for both beginners and experienced users of pandas. It includes detailed explanations of pandas concepts and functions, as well as tips and tricks for optimizing our code. At the end of this module, we’ll be able to effectively handle data using pandas.

Effective Data Manipulation with pandas

Look at the given DataFrame 'df' below:

```python
    a    b    c    
0   1    NaN  3    
1   4    5    6
2   7    NaN  NaN
```

How can we get the count of all `NaN` values in the DataFrame in Pandas?

Look at the given DataFrame ‘df’ below:
<pre><code class="language-python"> a b c 
0 1 NaN 3 
1 4 5 6
2 7 NaN NaN
</code></pre>
How can we get the count of all <code>NaN</code> values in the DataFrame in Pandas?

```python
     a    b    c    
0    1    NaN  NaN    
1    2    3    4
3    2    NaN  2
```

```python
     a    b    c    
1    2    3    4
```

```python
     a    b    c    
0    1    NaN  NaN    
2    NaN  NaN  NaN
3    5    NaN  6
```

```python
     a    b    c    
0    1    NaN  NaN    
2    NaN  NaN  NaN
```

Look at the given DataFrame `df` below:

```python
     a    b    c    
0    1    NaN  NaN    
1    2    3    4
2    NaN  NaN  NaN
3    5    NaN  6
```

What will be the output of the following command?
```python
df.dropna(axis = 0, how = 'any')
```


Look at the given DataFrame <code>df</code> below:
<pre><code class="language-python"> a b c 
0 1 NaN NaN 
1 2 3 4
2 NaN NaN NaN
3 5 NaN 6
</code></pre>
What will be the output of the following command?
<pre><code class="language-python">df.dropna(axis = 0, how = 'any')
</code></pre>

```python
     a    b    c    
0    1    4    1    
1    1    2    4
2    2    4    1
3    2    4    2 
```

```python
     a    b    c    
0    1    2    2    
1    1    2    4
2    1    2    2
3    2    2    2 
```

```python
     a    b    c    
0    1    4    1    
1    1    2    4
2    1.33 4    1
3    2    4    2 
```

```python
     a    b    c    
0    1    2    2    
1    1    2    4
2    1.33 2    2
3    2    2    2 
```

Look at the given DataFrame `df` below:

```python
     a    b    c    
0    1    NaN  NaN    
1    1    2    4
2    NaN  NaN  NaN
3    2    NaN  2 
```

What will be the output of the following command?

```python
df.fillna({'a':df['a'].mean(), 'b':df['b'].max(), 'c':df['c'].min()})
```


Look at the given DataFrame <code>df</code> below:
<pre><code class="language-python"> a b c 
0 1 NaN NaN 
1 1 2 4
2 NaN NaN NaN
3 2 NaN 2 
</code></pre>
What will be the output of the following command?
<pre><code class="language-python">df.fillna({'a':df['a'].mean(), 'b':df['b'].max(), 'c':df['c'].min()})
</code></pre>

Look at the given DataFrame `df` below:

```python
      col1  col2  col3    
r1    24    16    5    
r2    20    20    5    
r3    NaN   NaN   6
```

Which command will replace `NaN` with zeros?


Look at the given DataFrame <code>df</code> below:
<pre><code class="language-python"> col1 col2 col3 
r1 24 16 5 
r2 20 20 5 
r3 NaN NaN 6
</code></pre>
Which command will replace <code>NaN</code> with zeros?

**(True or False)** One of the parameters of `drop_duplicates()` functions is `keep`. Assuming that the value passed to `keep` is `last`, it means that “The last occurrence of the values are kept, and the rest are dropped.”


(True or False) One of the parameters of <code>drop_duplicates()</code> functions is <code>keep</code>. Assuming that the value passed to <code>keep</code> is <code>last</code>, it means that “The last occurrence of the values are kept, and the rest are dropped.”

Quiz yourself on missing and duplicated data in pandas series.

Introduction

Series Deep Dive

DataFrames

Manipulating Data

Conclusion

Quiz: Missing and Duplicated Data