How Data Travels
Learn how data travels through acquisition, processing, and storage to power modern apps and insights.
“Data engineering does not have an end state, but it’s a continual process of collecting, storing, processing, and analyzing data.”—Heather Miller
Imagine we take a photo on our phone and share it on a social media app. Within seconds, our friends can view, like, and leave comments on it. It all feels instant, but behind the scenes, something incredible just happened. That photo traveled a complex path: it moved from our phone, through the internet, into a system that stored, processed, and shared it.
This journey—from capturing data to sharing it in a useful form—is what we explore in this lesson. It's called data engineering workflow, and it's how data travels from one place to another while being cleaned, organized, and stored along the way.
In this lesson, we’re going to follow data on its journey—from the moment it’s created to the moment it’s ready to be used. This journey is at the heart of data engineering. It involves a series of important steps that ensure raw, messy data becomes clean, organized, and valuable.
Data engineering workflow
We’ll break this journey into three major steps:
Data acquisition: Data is collected from its source.
Data processing: Data is cleaned, structured, and transformed.
Data storage: Data is stored safely and efficiently for later use.
Let’s begin this journey by exploring the very first step: how we catch the data right where it starts.
Step 1: Data acquisition
Every journey starts at the beginning, and for data, that beginning is acquisition. This is where we collect raw data from its original source, ...