AI-powered learning
Save this course
Data Engineering Foundations in Python
Gain insights into data engineering foundations, explore data life cycle stages, and delve into creating data pipelines using Python, Kafka, PySpark, Airflow, and dbt.
4.6
46 Lessons
2 Projects
7h
Updated 5 months ago
Join 2.9 million developers at
Join 2.9 million developers at
LEARNING OBJECTIVES
- An understanding of the data engineering life cycle
- Familiarity with the cloud data warehouse and data modeling techniques
- Hands-on experience with data engineering tools such as GCP, Airflow, Spark, and dbt
- The ability to build data pipelines from scratch in Python
Learning Roadmap
2.
Data Team Structure
Data Team Structure
Get started with the essentials of data team roles and effective team structures.
3.
Data Engineering Life Cycle
Data Engineering Life Cycle
5 Lessons
5 Lessons
Master the steps to implement and manage data engineering life cycle stages using Google Cloud.
4.
Cloud Data Architecture
Cloud Data Architecture
6 Lessons
6 Lessons
Grasp the fundamentals of cloud data evolution, service models, architectures, and best practices for efficiency and security.
5.
Data Ingestion
Data Ingestion
9 Lessons
9 Lessons
Take a closer look at methods and tools for efficient data ingestion using Python.
6.
Data Modeling
Data Modeling
8 Lessons
8 Lessons
Focus on mastering data modeling and SQL for effective database and analytical operations.
7.
Data Orchestration
Data Orchestration
8 Lessons
8 Lessons
Master data orchestration using tools like Airflow, Dagster, and dbt to enhance workflows.
8.
Data Quality
Data Quality
4 Lessons
4 Lessons
Solve problems in measuring and maintaining data quality with schema validation and testing.
Certificate of Completion
Showcase your accomplishment by sharing your certificate of completion.
Complete more lessons to unlock your certificate
Developed by MAANG Engineers
ABOUT THIS COURSE
Data engineering is currently one of the most in-demand fields in data and technology. It intersects software engineering, DataOps, data architecture, data management, and security. Data engineers, such as analysts and data scientists, lay the foundation to serve data for consumers.
In this course, you will learn the foundation of data engineering, covering different parts of the entire data life cycle: data warehouse, ingestion, transformation, orchestration, etc. You will also gain hands-on experience building data pipelines using different techniques such as Python, Kafka, PySpark, Airflow, dbt, and more.
By the end of this course, you will have a holistic understanding of data engineering and be able to build your data pipelines to serve data for various consumers.
ABOUT THE AUTHOR
Xiaoxu Gao
I'm a Data Engineer, Software Engineer, and a Passionate Tech Mentor. I help people advance their careers through my knowledge and empathy.
Trusted by 2.9 million developers working at companies
A
Anthony Walker
@_webarchitect_
E
Evan Dunbar
ML Engineer
S
Software Developer
Carlos Matias La Borde
S
Souvik Kundu
Front-end Developer
V
Vinay Krishnaiah
Software Developer
Built for 10x Developers
No Passive Learning
Learn by building with project-based lessons and in-browser code editor
Personalized Roadmaps
The platform adapts to your strengths & skills gaps as you go
Future-proof Your Career
Get hands-on with in-demand skills
AI Code Mentor
Write better code with AI feedback, smart debugging, and "Ask AI"
MAANG+ Interview Prep
AI Mock Interviews simulate every technical loop at top companies
Free Resources
