0% completed

All LessonsFree Lessons (3)

Getting Started

Data Team Structure

Data Team Roles Data Team Structure: Embedded or Centralized?Quiz: Data Team Structure

Data Engineering Life Cycle

Set Up the Google Cloud

Introduction to Data Engineering Life Cycle

Ingestion, Transformation, and Visualization

Storage and Infrastructure

Quiz: Data Engineering Life Cycle

Cloud Data Architecture

Evolution of Cloud Data Architecture Cloud Service Models Cloud Data Architectures: Lambda, Kappa, and Data Warehouse Cloud Data Architectures: Data Lake and Data Mesh Patterns of Good Cloud Data Architecture Quiz: Cloud Data Architecture

Data Ingestion

Batch Ingestion Stream Ingestion Push vs. Pull Ingestion Methods—SFTP, API, and Object Storage Ingestion Methods—CDC Ingestion Methods—Streaming Platform Ingest with pandas Ingest with PySpark Quiz: Data Ingestion

Data Modeling

Introduction to Data Modeling Kimball’s Dimensional Modeling Steps of Dimensional Modeling DDL vs. DML Query Life Cycle Challenge: BigQuery SQL Solution: BigQuery SQL Quiz: Data Modeling

Data Orchestration

What Is Data Orchestration?Airflow Basics Airflow DAG Design Challenge: Build an FXRate Reporting Pipeline with Airflow Solution: Build an FXRate Reporting Pipeline with Airflow Orchestration Tool: Dagster Orchestration Tool: dbt Quiz: Data Orchestration

Project

Mastering Airflow: Building an ETL Pipeline

Data Quality

Data Quality Measurement Data Schema: Avro and Protobuf Manage Data Quality in dbt Quiz: Data Quality

Mini Project

Build an End-to-End Data Pipeline for Formula 1 Analysis

Epilogue

The Way Forward

Appendix

Additional Resources

Data Engineering Foundations in Python/

...

/

Quiz: Data Engineering Life Cycle

Quiz: Data Engineering Life Cycle

Test your knowledge of data engineering life cycle.

We'll cover the following...

Technical Quiz

1.

Which task is not part of the data engineering life cycle?

A.

Building CI/CD for data pipelines

B.

Applying IAM policies to data warehouse

C.

Setting up the Tableau server

D.

None of the above

1 / 7

...