HomeCoursesData Science in Production: Building Scalable Model Pipelines

AI-powered learning

Save

Data Science in Production: Building Scalable Model Pipelines

Gain insights into building scalable data and model pipelines, explore different cloud environments, delve into streaming workflows, and discover essential tools for creating real-time data products.

4.6

110 Lessons

4 Cloud Labs

Updated 1 month ago

Join 3 million developers at

LEARNING OBJECTIVES

Identify cloud tools (AWS/GCP) to build scalable batch and streaming model workflows
Set up remote Jupyter on AWS EC2 with SSH and firewall rules for cloud development
Load and automate dataset ingestion into Pandas from BigQuery and Kaggle using Python APIs
Build and evaluate prototype models using scikit-learn, PySpark MLlib, and Keras (ROC/AUC, errors)
Deploy models as web endpoints using Flask, Gunicorn, Heroku, and Dash for real-time predictions
Implement serverless model endpoints with GCP Cloud Functions and AWS Lambda using S3/GCS and IAM
Construct batch and streaming pipelines with PySpark, Apache Beam/Dataflow, Kafka, and PubSub

Learning Roadmap

110 Lessons16 Quizzes4 Cloud Labs

Introduction to Building Scalable Model Pipelines

Get familiar with building scalable data pipelines, using Python, clouds, coding environments.

Course Overview

Applied Data Science

Python for Scalable Compute

Cloud Environments

Coding Environments

Quiz - Data Science Preliminary Concepts

🛑 Important Note!

Introduction to Datasets

- BigQuery to Pandas

- Kaggle to Pandas

Quiz - Importing Datasets

Prototype Models

- Linear Regression

- Logistic Regression

- Keras Regression

Automated Feature Engineering

Quiz - Prototype Modeling

Conclusion : Data science models, tools and environments

Models as Web Endpoints

Solve challenges with deploying machine learning models as scalable web endpoints.

Introduction to Models as Web Endpoints

Quiz - Web Services and Persistent Models

Deploying a Web Endpoint

- Gunicorn

- Heroku

Interactive Web Services with Dash

Quiz - Web Deployments

Conclusion : Models as Web Endpoints

Models as Serverless Functions

17 Lessons

Go hands-on with deploying machine learning models using serverless functions on AWS and GCP.

Create an Echo Function in Lambda

Cloud Lab

Premium Plus

Working with S3 in Lambda

Cloud Lab

Premium Plus

Working with API in Lambda

Cloud Lab

Premium Plus

Containers for Reproducible Models

9 Lessons

Grasp the fundamentals of using containers to ensure reproducible, scalable model deployments.

Working with AWS Container Registry

Cloud Lab

Premium Plus

Workflow Tools for Model Pipelines

10 Lessons

Take a closer look at managing and automating model pipelines with workflow tools like Airflow.

PySpark for Batch Pipelines

25 Lessons

Follow the process of building scalable model pipelines using PySpark in cloud environments.

Cloud Dataflow for Batch Modeling

8 Lessons

Master the creation and execution of scalable Cloud Dataflow pipelines for batch modeling.

Streaming Model Workflows

10 Lessons

Learn how to use streaming platforms for scalable real-time machine learning pipelines.

Course Conclusion

Look at the evolving journey in data science, emphasizing continuous learning and practice.

Thank You

Certificate of Completion

Showcase your accomplishment by sharing your certificate of completion.

Developed by MAANG Engineers

ABOUT THIS COURSE

The goal of this course is to provide you with a set of tools that can be used to build predictive model services for product teams. In this course, you’ll start by covering the different cloud environments and tools for building scalable data and model pipelines. You’ll then learn the different data sets and types of models that will be used heavily in everyday production. Throughout the course, you’ll have plenty of exercises and challenges to get you comfortable working with the diverse toolset. Lastly, you’ll explore streaming model workflows which is crucial for building real-time data pipelines that move data between different components in a cloud environment. After working through this course, you will have gained valuable hands-on experience with many of the tools needed to build data products. You will also have a better understanding of how to build scalable machine learning pipelines in a cloud environment.

ABOUT THE AUTHOR

Ben Weber

Data Science at Zynga, UCSC Alum

Learn more about Ben

Trusted by 3 million developers working at companies

These are high-quality courses. Trust me the price is worth it for the content quality. Educative came at the right time in my career. I'm understanding topics better than with any book or online video tutorial I've done. Truly made for developers. Thanks

Anthony Walker

@_webarchitect_

Just finished my first full #ML course: Machine learning for Software Engineers from Educative, Inc. ... Highly recommend!

Evan Dunbar

ML Engineer

You guys are the gold standard of crash-courses... Narrow enough that it doesn't need years of study or a full blown book to get the gist, but broad enough that an afternoon of Googling doesn't cut it.

Software Developer

Carlos Matias La Borde

I spend my days and nights on Educative. It is indispensable. It is such a unique and reader-friendly site

Souvik Kundu

Front-end Developer

Your courses are simply awesome, the depth they go into and the breadth of coverage is so good that I don't have to refer to 10 different websites looking for interview topics and content.

Vinay Krishnaiah

Software Developer

Course

The Hacker's Guide to Scaling Python

Learn about Python's scalability, concurrency, and distribution via CPU scaling, event loops, queue-based distribution, and building REST APIs to optimize and deploy high-performance applications.

9 h

advanced

Course

Applied Machine Learning: Deep Learning for Industry

Gain insights into industry-level machine learning by applying advanced TensorFlow and Python techniques. Explore efficient methods for creating scalable models, from data pipelines to deployment and inference.

3 h

advanced

Course

Feature Engineering for Machine Learning

30 m

advanced

Course

Hands-On Generative Adversarial Networks with PyTorch

In this GAN course, learn GAN fundamentals and PyTorch. Explore DCGANs, conditional GANs, image translations, and text-to-image synthesis to master advanced skills for real-world applications.

16 h

advanced

Course

Web Security and Access Management: JWT, OAuth2 & OpenId Connect

Gain insights into JWT, OAuth2, and OpenID Connect. Learn about HTTPS, encryption, and handshake techniques, and explore foundational concepts in web API security for enhanced application protection.

5 h

intermediate

Course

Web Application Security for the Everyday Software Engineer

Gain insights into enforcing web app security best practices, such as HTTPS, defending against XSS and clickjacking, managing HTTP cookies, and warding off DDoS attacks.

4 h

intermediate

Course

Building Full-Stack Web Applications with AdonisJs

Gain insights into AdonisJs fundamentals. Delve into routes, controllers, middleware, and hooks. Discover practical examples, illustrations, and quizzes to confidently build your full-stack applications.

intermediate

Course

The Hacker's Guide to Scaling Python

advanced

9 hour

Course

Applied Machine Learning: Deep Learning for Industry

advanced

3 hour

Course

Feature Engineering for Machine Learning

advanced