Customizing Foundation Models
Explore techniques to customize foundation models in AWS, from prompt engineering and retrieval augmentation to fine-tuning and distillation. Understand when to apply each method based on task requirements, cost, and performance. Learn how to run fine-tuning jobs, evaluate custom models, and choose hosting options to effectively deploy scalable and cost-efficient AI solutions in production.
A foundation model chosen in Amazon Bedrock or SageMaker JumpStart rarely behaves exactly as a business needs on the first call. It may answer in the wrong format, miss the company’s terminology, ignore a classification scheme, or cost too much per request at production volume. Customization closes that gap. The techniques range from changing only the text sent to the model, through grounding it with retrieved documents, to changing the model’s weights with a training job.
The exam tests whether you can pick the lightest technique that meets the requirement. Prompting has no training cost and changes in minutes. Fine-tuning, continued pre-training, and distillation change or replace the model, so they need data, a training job, and a hosting decision. Customization is done when the technique matches the gap, the evidence (evaluation results or training and validation loss) shows that the gap closed, and the per-call cost fits the budget.
The diagram below shows where each technique acts.
Task-specific prompt engineering
temperature and maxTokens, covered in the Amazon Bedrock Foundations and Model Selection lesson, control how the model samples. Prompt engineering controls what it is asked.
Zero-shot and few-shot prompting
A zero-shot prompt gives only the instruction and the input: “Classify this headline as positive, negative, or neutral.” It works when the task is common and the label set is obvious. Few-shot prompting, also called in-context learning, adds a handful of examples, where each shot is a paired example input and its desired output. The examples calibrate the model’s output format and the boundaries between labels without any training.
Few-shot prompting is the first thing to try when a zero-shot prompt returns the right kind of answer in an inconsistent format, or confuses two similar labels. Its cost is visible on every call: each example adds input tokens, and at millions of requests a long few-shot prompt can cost more than a one-time fine-tuning job.