Learn about different cloud environments and tools for building scalable data and model pipelines.

Become a Data Scientist

In this module, we’ll cover different cloud environments and tools for building scalable data and model pipelines. We’ll learn the different data sets and types of models that will be used heavily in everyday production. Throughout the module, we’ll have plenty of exercises and challenges to get us comfortable working with the diverse toolset. Lastly, we’ll explore streaming model workflows that are crucial for building real-time data pipelines.
By the end of this module, we'll have a better understanding of how to build scalable machine learning pipelines in a cloud environment.

Data Science in Production: Building Scalable Model Pipelines

Containers are great to make sure that your analyses and models are reproducible across different environments. While containers are useful for keeping dependencies clean on a single machine, the main benefit is that they enable data scientists to write model endpoints without worrying about how the container will be hosted. This separation of concerns makes it easier to partner with engineering teams to deploy models to production. Using the approaches shown in this chapter, data and applied science teams can also own the deployment of models to production.

Introduction to Building Scalable Model Pipelines

Models as Web Endpoints

Models as Serverless Functions

Containers for Reproducible Models

Workflow Tools for Model Pipelines

PySpark for Batch Pipelines

Cloud Dataflow for Batch Modeling

Streaming Model Workflows

Conclusion

Conclusion: Containers as Reproducible Models