Search⌘ K

Data Science in Production: Building Scalable Model Pipelines

0%

1.Introduction to Building Scalable Model Pipelines

Course Overview Applied Data Science Python for Scalable Compute Cloud Environments Coding Environments Quiz - Data Science Preliminary Concepts 🛑 Important Note!Introduction to Datasets - BigQuery to Pandas - Kaggle to Pandas Quiz - Importing Datasets Prototype Models - Linear Regression - Logistic Regression - Keras Regression Automated Feature Engineering Quiz - Prototype Modeling Conclusion : Data science models, tools and environments

2.Models as Web Endpoints

Introduction to Models as Web Endpoints Web Services Echo Service Model Persistence Model Endpoints Quiz - Web Services and Persistent Models Deploying a Web Endpoint - Gunicorn - Heroku Interactive Web Services with Dash Quiz - Web Deployments Conclusion : Models as Web Endpoints

3.Models as Serverless Functions

Introduction to Models as Serverless Functions Managed Services Cloud Functions (GCP)- Echo Service - Cloud Storage (GCS)- Model Function - Keras Model - Access Control - Model Refreshes Quiz - Managed Services Lambda Functions (AWS)- Echo Function - Simple Storage Service (S3)- Model Function - API Gateway Quiz - Lambda Functions Conclusion

Cloud Lab

Create an Echo Function in Lambda

Cloud Lab

Working with S3 in Lambda

Cloud Lab

Working with API in Lambda

4.Containers for Reproducible Models

Introduction to Containers as Reproducible Models Docker Orchestration - AWS Container Registry (ECR)- AWS Container Service (ECS)- Load Balancing Kubernetes on GCP Quiz - Containers Conclusion: Containers as Reproducible Models

Cloud Lab

Working with AWS Container Registry

5.Workflow Tools for Model Pipelines

Introduction to Workflow Tools for Model Pipelines Sklearn Workflow Cron - Cloud Cron Quiz - Workflows and Scheduling Workflow Tools - Apache Airflow - Managed Airflow Quiz - Workflow Tools Conclusion : Workflow Tools for Model Pipelines

6.PySpark for Batch Pipelines

Introduction to PySpark for Batch Pipelines Spark Environments - Spark Clusters - Databricks Community Edition Staging Data Quiz - Spark Environments A PySpark Primer - Persisting Dataframes - Converting Dataframes - Transforming Data - Pandas UDFs - Best Practices Quiz - PySpark MLlib Batch Pipeline - Model Application Distributed Deep Learning Distributed Feature Engineering Quiz - Machine Learning in PySpark GCP Model Pipeline - BigQuery Export - GCP Credentials - Model Pipeline Productizing PySpark Quiz - GCP Model Pipeline Conclusion : Working tools for model pipelines

7.Cloud Dataflow for Batch Modeling

Introduction to Cloud Dataflow and Batch Modeling Apache Beam Batch Model Pipeline - Model Training - BigQuery Publish - Datastore Publish Quiz - Dataflow and Batch Modeling Conclusion : Cloud Dataflow for Batch Modeling

8.Streaming Model Workflows

Introduction to Streaming Model Workflows Spark Streaming - Apache Kafka - Sklearn Streaming 1 - Sklearn Streaming 2 Dataflow Streaming - PubSub - Natality Streaming Quiz - Streaming Model Workflows Conclusion : Streaming model workflows

9.Course Conclusion

Data Science in Production: Building Scalable Model Pipelines

Create an Echo Function in Lambda

Takes 30 mins

AWS Lambda is a serverless, event-driven compute service that runs code for virtually any application or back-end service without provisioning or managing servers.

In this Cloud Lab, you’ll create a simple Lambda function that echoes the input provided to it.

Lab tasks

Create a Function
Test the Function
Clean Up