Example of a serverless ML Pipeline in AWS
Explore building a serverless machine learning pipeline in AWS by integrating Lambda functions, including a Dockerized ML model, with AWS Step Functions for orchestration and EventBridge for event-driven triggers. Understand how to deploy and manage an end-to-end workflow that transforms and processes ML predictions within the AWS cloud.
We'll cover the following...
This last example will be special, as it will bring together all the components we've learned so far. We'll implement this with AWS Step Functions. It’s an orchestration service used to build pipelines in the cloud.
In this example, we will deploy a service that consists of the following:
-
ML model: A previously trained ML model wrapped in a Docker image and deployed to Lambda using our custom image in order to serve model predictions.
-
AWS Step Functions pipeline: This small pipeline will integrate our model prediction with another Lambda function.
-
AWS EventBridge: We’ll integrate another Lambda function outside our
SF pipeline Step Functions Pipeline -
Two additional Lambda functions: These are variations of functions we’ve already deployed before, but with a few tweaks. One of these functions will be integrated in our Step Functions pipeline, and the other will be invoked through the AWS EventBridge service. Together, they will constitute an entire mini-model pipeline.
