Unveiling the Numbers

Data doesn’t speak evenly; some details carry more weight than others. At this stage, our task is to recognize the elements that matter most within the dataset. By doing so, we are preparing ourselves for insights that are not only clear, but also actionable.

This brings us to the heart of exploratory data analysis (EDA). Having already cleaned, organized, and taken a first look at the data, we’re now ready to go deeper. EDA is where we pause to examine each variable more closely, uncover its unique characteristics, and notice patterns or unusual values that might shape our understanding.

We begin with curiosity, asking simple but powerful questions:

• What does each variable reveal on its own?

• Which values dominate, and which are rare?

• Are there outliers or unexpected shapes in the data?

At this stage, we’re not rushing to conclusions; instead, we’re training ourselves to recognize what matters. This habit of close observation is what makes EDA such a critical step in the analytical process.

What is EDA?

Every dataset holds a story, but that story isn’t always immediately clear. Before we create charts, run summaries, or share insights, we need to understand the data’s structure, patterns, and oddities. That’s where exploratory data analysis (EDA) comes in. It’s how data analysts get familiar with the data, uncover what’s worth highlighting, and spot anything that might affect the integrity of the analysis.

The key steps of exploratory data analysis

Let’s break down the key steps of exploratory data analysis, and what we actually do when we explore data, from summarizing distributions to spotting relationships.

1. Get to know the data basics: This step involves identifying the dataset’s rows (records) and columns (variables), and reviewing basic summary statistics (MIN, MAX, COUNT) to understand its structure and contents. This is the first step in EDA, and we’ve already covered these skills in earlier lessons.

2. Explore variables one at a time (univariate analysis): Next, we examine each variable individually. We want to understand its distribution, common values, and any oddities like outliers or missing data.

3. Look at relationships between variables (bivariate and multivariate analysis): Then, we study how variables interact. Are some variables correlated? Are there patterns when we group data by categories? This step uncovers connections that can be important for deeper analysis.

4. Visualize the data: Visualization plays a huge role in EDA. Charts like histograms, scatterplots, and bar plots help us see patterns, spot anomalies, and communicate findings clearly.

5. Iterate between exploring and cleaning: As we explore, we often find data issues, like missing values, inconsistencies, or errors. We then clean or transform the data, and revisit the exploration. This iterative cycle continues until the data is well understood and ready.

6. Use insights to guide deeper analysis: Finally, the insights from EDA help us ask better questions, select features, and build more effective analyses.

Before we jump into comparing variables or building visuals, it helps to slow down and focus on one variable at a time. This step is like turning a spotlight on each column in our ...