Learn about different cloud environments and tools for building scalable data and model pipelines.

Become a Data Scientist

In this module, we’ll cover different cloud environments and tools for building scalable data and model pipelines. We’ll learn the different data sets and types of models that will be used heavily in everyday production. Throughout the module, we’ll have plenty of exercises and challenges to get us comfortable working with the diverse toolset. Lastly, we’ll explore streaming model workflows that are crucial for building real-time data pipelines.
By the end of this module, we'll have a better understanding of how to build scalable machine learning pipelines in a cloud environment.

Data Science in Production: Building Scalable Model Pipelines

GCP provides a fully-managed streaming platform called ___________.

GCP provides a fully-managed streaming platform called ___________.

While open-source Kafka is a hosted solution for message streaming, some cloud providers now offer fully-managed versions of Kafka, such as Amazon’s MSK offering.

While open-source Kafka is a hosted solution for message streaming, some cloud providers now offer fully-managed versions of Kafka, such as Amazon’s MSK offering.

When setting up Kafka, we’ll need to spawn ___________ separate processes to run dependencies, start the Kafka service, and create a new topic for publishing messages.

When setting up Kafka, we’ll need to spawn ___________ separate processes to run dependencies, start the Kafka service, and create a new topic for publishing messages.

Some of the GCP systems that work with Dataflow, such as BigQuery, can be used _____________________.

Some of the GCP systems that work with Dataflow, such as BigQuery, can be used _____________________.

In Kafka, you can publish and subscribe to a topic directly, while in PubSub consumers subscribe to subscriptions rather than directly subscribing to topics.

In Kafka, you can publish and subscribe to a topic directly, while in PubSub consumers subscribe to subscriptions rather than directly subscribing to topics.

A quiz on different streaming model workflows. 

Introduction to Building Scalable Model Pipelines

Models as Web Endpoints

Models as Serverless Functions

Containers for Reproducible Models

Workflow Tools for Model Pipelines

PySpark for Batch Pipelines

Cloud Dataflow for Batch Modeling

Streaming Model Workflows

Conclusion

Quiz - Streaming Model Workflows