HomeCoursesData Engineering Foundations in Python

Updated 27 Aug

AI-powered learning

Save

Data Engineering Foundations in Python

Name: Data Engineering Foundations in Python
Price: 59 USD
Rating: 4.69 (101 reviews)
Author: Xiaoxu Gao

Gain insights into data engineering foundations, explore data life cycle stages, and delve into creating data pipelines using Python, Kafka, PySpark, Airflow, and dbt.

4.7

46 lessons

2 projects

1 AI mock interview

Created by Xiaoxu Gao

What's included

46 lessons across 10 modules
7 quizzes with instant feedback
2 projects in preconfigured environments
1 AI mock interview with scoring rubrics
AI Tutor on every lesson

Join 3.1 million developers at

What You'll Learn

An understanding of the data engineering life cycle
Familiarity with the cloud data warehouse and data modeling techniques
Hands-on experience with data engineering tools such as GCP, Airflow, Spark, and dbt
The ability to build data pipelines from scratch in Python

Learning Roadmap

46 Lessons1 Project7 Quizzes

Getting Started

Get familiar with essential data engineering skills, including cloud architectures and pipeline orchestration.

Introduction

Data Team Structure

Get started with the essentials of data team roles and effective team structures.

Data Team Roles

Data Team Structure: Embedded or Centralized?

Quiz: Data Team Structure

Data Engineering Life Cycle

5 Lessons

Master the steps to implement and manage data engineering life cycle stages using Google Cloud.

Cloud Data Architecture

6 Lessons

Grasp the fundamentals of cloud data evolution, service models, architectures, and best practices for efficiency and security.

Data Ingestion

9 Lessons

Take a closer look at methods and tools for efficient data ingestion using Python.

Data Modeling

8 Lessons

Focus on mastering data modeling and SQL for effective database and analytical operations.

Data Orchestration

8 Lessons

Master data orchestration using tools like Airflow, Dagster, and dbt to enhance workflows.

Mastering Airflow: Building an ETL Pipeline

Project

Premium

Data Quality

4 Lessons

Solve problems in measuring and maintaining data quality with schema validation and testing.

Build an End-to-End Data Pipeline for Formula 1 Analysis

Project

Epilogue

Look at the fundamentals of data engineering, future paths, and mastering essential tools.

The Way Forward

10.

Appendix

Explore additional GCP resources and billing management for efficient cloud utilization.

Additional Resources

Data Engineering Fundamentals

Mock Interview

Premium

Certificate of Completion

Showcase your accomplishment by sharing your certificate of completion.

About this course

WHY I WROTE THIS COURSE

Data engineering is currently one of the most in-demand fields in data and technology. It intersects software engineering, DataOps, data architecture, data management, and security. Data engineers, such as analysts and data scientists, lay the foundation to serve data for consumers. In this course, you will learn the foundation of data engineering, covering different parts of the entire data life cycle: data warehouse, ingestion, transformation, orchestration, etc. You will also gain hands-on experience building data pipelines using different techniques such as Python, Kafka, PySpark, Airflow, dbt, and more. By the end of this course, you will have a holistic understanding of data engineering and be able to build your data pipelines to serve data for various consumers.

ABOUT THE AUTHOR

Xiaoxu Gao

I'm a Data Engineer, Software Engineer, and a Passionate Tech Mentor. I help people advance their careers through my knowledge and empathy.

Learn more about Xiaoxu

Trusted by 3.1 million developers working at companies

These are high-quality courses. Trust me the price is worth it for the content quality. Educative came at the right time in my career. I'm understanding topics better than with any book or online video tutorial I've done. Truly made for developers. Thanks

Anthony Walker

@_webarchitect_

Just finished my first full #ML course: Machine learning for Software Engineers from Educative, Inc. ... Highly recommend!

Evan Dunbar

ML Engineer

You guys are the gold standard of crash-courses... Narrow enough that it doesn't need years of study or a full blown book to get the gist, but broad enough that an afternoon of Googling doesn't cut it.

Software Developer

Carlos Matias La Borde

I spend my days and nights on Educative. It is indispensable. It is such a unique and reader-friendly site

Souvik Kundu

Front-end Developer

Your courses are simply awesome, the depth they go into and the breadth of coverage is so good that I don't have to refer to 10 different websites looking for interview topics and content.

Vinay Krishnaiah

Software Developer

Course

Mastering Apache Kafka

Learn about Apache Kafka's architecture, client APIs, stream processing, and connectors. Build real-time data streaming and event-driven applications while exploring Kafka's ecosystem.

7 h

intermediate

Course

Data Science in Production: Building Scalable Model Pipelines

Gain insights into building scalable data and model pipelines, explore different cloud environments, delve into streaming workflows, and discover essential tools for creating real-time data products.

8 h

advanced

Course

Cracking the Google Associate Cloud Engineer Certification

Become a cloud pro with strategic GCP skills. Get hands-on experience, understand "Why GCP?", and prepare with a practice exam. No setup, no cleanup, no hassle.

4 h

beginner

Course

Introduction to Big Data and Hadoop

Delve into Big Data essentials, explore data types, and gain insights into Hadoop components like YARN, MapReduce, HDFS, and Spark. Discover foundations to excel in the growing Big Data field.

10 h

beginner

Course

An Introduction to Spark

Gain insights into Spark, its architecture, application lifecycle, and APIs. Delve into data frames, datasets, and Spark SQL to effectively manage and query big data.

2 h 50 m

beginner

Course

From Pandas to PySpark DataFrame

Gain insights into enhancing Python data processing with PySpark. Delve into reading, transforming, aggregating data, and creating user-defined functions, boosting efficiency with Apache Spark.

3 h 3 m

intermediate

Course

Mastering Apache Kafka

intermediate

7 hour

Course

Data Science in Production: Building Scalable Model Pipelines

advanced

8 hour

Course

Cracking the Google Associate Cloud Engineer Certification