Search⌘ K
AI Features
Home
Courses
Grokking the AWS Certified Machine Learning Engineer - Associate

MLOps for Generative AI

Explore how to operationalize generative AI applications on AWS by managing version-controlled prompts, fine-tuned model deployment, and knowledge base refresh workflows. Understand testing approaches for non-deterministic outputs, safe promotion and rollback strategies, and automation with AWS services like Bedrock Prompt Management, EventBridge, and Step Functions. Gain practical insights into scalable, production-ready generative AI lifecycle management.

We'll cover the following...

The CI/CD Pipelines and Deployment Strategies lesson moved code and model artifacts to a SageMaker AI endpoint. A generative AI application changes in more places. A one-line edit to a prompt can change every answer. A new fine-tuned model replaces the weights behind the same API call. A policy document added to Amazon S3 changes what a retrieval-augmented generation (RAG) application can cite, but only after the knowledge base is synced.

Each of these is a release, and each needs the same three things as a model release: a version that can be referenced, a test that decides whether the version ships, and a way back. A release is done when production points at an identified version, the test results that approved it are recorded, and the previous version is one configuration change away.

The diagram below shows the three kinds of change and the release path they share.

Three generative AI changes, one release path
Three generative AI changes, one release path

Managing prompts with Amazon Bedrock Prompt Management

Prompts hardcoded in application code are released with the code, can’t be compared, and can’t be rolled back on their own. Amazon Bedrock Prompt Management stores them as AWS resources with their own identity, versions, and permissions.

Prompts, variables, and variants

A prompt in Prompt Management holds the message template, the model or inference profile to run it on, and the inference parameters, such as maxTokens and temperature. With the CHAT template type, it can also hold a system prompt, earlier conversation turns, and tool definitions. Placeholders in double curly braces, such as {{ticket_text}}, are variables that the caller fills in at run time.

A variant is an alternative configuration of the prompt: a different message, model, or set of inference parameters. In the prompt builder, Compare variants runs up to three variants side by side with the same test values. Saving one with Save as draft replaces the working draft and deletes the other variants. Variants are an experimentation tool, not parallel production versions.

Versions and promotion

Saving a prompt creates or updates its DRAFT version. When the draft is ready, CreatePromptVersion takes a snapshot of it. Versions are numbered from 1, and each has its own ARN ending in :1, :2, and so ...