Missing Data
Explore techniques for handling missing data in datasets using Python. Learn how to assess missing data reasons, use statistical values or models to fill gaps, and apply Pandas functions to track or drop missing entries.
We'll cover the following...
What is missing data? #
So, you have read in your data only to discover that some values are missing! What do you do?
First, you should try to understand why your data points are missing. Are they missing at random or not? Data that is missing at random could be removed with a large enough dataset. If your data is missing for a reason, perhaps missing means zero? Or is it a strong signal of a sensor malfunction? These non-random missing values should be fixed or leveraged. For example, set them to zero if they should be zero.
Second, you should consider how much data you have relative to how many rows have missing ...