Data Science in Production: Building Scalable Model Pipelines

0%

1.Introduction to Building Scalable Model Pipelines

Course Overview Applied Data Science Python for Scalable Compute Cloud Environments Coding Environments Quiz - Data Science Preliminary Concepts 🛑 Important Note!Introduction to Datasets - BigQuery to Pandas - Kaggle to Pandas Quiz - Importing Datasets Prototype Models - Linear Regression - Logistic Regression - Keras Regression Automated Feature Engineering Quiz - Prototype Modeling Conclusion : Data science models, tools and environments

2.Models as Web Endpoints

Introduction to Models as Web Endpoints Web Services Echo Service Model Persistence Model Endpoints Quiz - Web Services and Persistent Models Deploying a Web Endpoint - Gunicorn - Heroku Interactive Web Services with Dash Quiz - Web Deployments Conclusion : Models as Web Endpoints

3.Models as Serverless Functions

Introduction to Models as Serverless Functions Managed Services Cloud Functions (GCP)- Echo Service - Cloud Storage (GCS)- Model Function - Keras Model - Access Control - Model Refreshes Quiz - Managed Services Lambda Functions (AWS)- Echo Function - Simple Storage Service (S3)- Model Function - API Gateway Quiz - Lambda Functions Conclusion

Cloud Lab

Create an Echo Function in Lambda

Cloud Lab

Working with S3 in Lambda

Cloud Lab

Working with API in Lambda

4.Containers for Reproducible Models

Introduction to Containers as Reproducible Models Docker Orchestration - AWS Container Registry (ECR)- AWS Container Service (ECS)- Load Balancing Kubernetes on GCP Quiz - Containers Conclusion: Containers as Reproducible Models

Cloud Lab

Working with AWS Container Registry

5.Workflow Tools for Model Pipelines

Introduction to Workflow Tools for Model Pipelines Sklearn Workflow Cron - Cloud Cron Quiz - Workflows and Scheduling Workflow Tools - Apache Airflow - Managed Airflow Quiz - Workflow Tools Conclusion : Workflow Tools for Model Pipelines

6.PySpark for Batch Pipelines

Introduction to PySpark for Batch Pipelines Spark Environments - Spark Clusters - Databricks Community Edition Staging Data Quiz - Spark Environments A PySpark Primer - Persisting Dataframes - Converting Dataframes - Transforming Data - Pandas UDFs - Best Practices Quiz - PySpark MLlib Batch Pipeline - Model Application Distributed Deep Learning Distributed Feature Engineering Quiz - Machine Learning in PySpark GCP Model Pipeline - BigQuery Export - GCP Credentials - Model Pipeline Productizing PySpark Quiz - GCP Model Pipeline Conclusion : Working tools for model pipelines

7.Cloud Dataflow for Batch Modeling

Introduction to Cloud Dataflow and Batch Modeling Apache Beam Batch Model Pipeline - Model Training - BigQuery Publish - Datastore Publish Quiz - Dataflow and Batch Modeling Conclusion : Cloud Dataflow for Batch Modeling

8.Streaming Model Workflows

Introduction to Streaming Model Workflows Spark Streaming - Apache Kafka - Sklearn Streaming 1 - Sklearn Streaming 2 Dataflow Streaming - PubSub - Natality Streaming Quiz - Streaming Model Workflows Conclusion : Streaming model workflows

9.Course Conclusion

Data Science in Production: Building Scalable Model Pipelines/

Working with API in Lambda

Working with API in Lambda

Takes 60 mins

Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale.

In this Cloud Lab, you’ll learn to add an API Gateway trigger to a Lambda function.

Lab tasks

Create a Lambda Function
Add an API Gateway Trigger
Clean Up