Experiment Tracking and Model Lineage
Explore how to implement experiment tracking and model lineage in Amazon SageMaker. Understand methods to record parameters, metrics, and artifacts, compare training runs, and trace the complete lifecycle from data to deployed models. This lesson helps you ensure reproducibility and regulatory compliance in machine learning production workflows.
We'll cover the following...
Pipelines automate execution. The Model Registry governs promotion. But neither answers the critical question: Why did this model perform differently from the last one? Experiment tracking fills this gap by systematically recording parameters, metrics, and artifacts across every training run, enabling reproducibility and structured comparison.
Model lineage extends this further. It constructs a directed graph connecting datasets, processing jobs, training jobs, and deployed endpoints into a single, queryable provenance chain. Together, these capabilities satisfy regulatory requirements in financial services, healthcare, and any domain where model decisions carry legal weight.
SageMaker provides two tracking approaches:
Native SageMaker Experiments, which is tightly integrated with the service ecosystem.
A managed MLflow integration for teams with existing MLflow workflows.
Both feed into the same Model Registry and lineage graph, so the choice affects the developer experience without constraining downstream deployment patterns.
This lesson progresses through native tracking mechanics, MLflow integration architecture, run comparison workflows, and lineage queries, and concludes with a decision framework for selecting the right approach.
Native SageMaker Experiments for tracking
SageMaker Experiments organizes tracking data into a clear structure designed for enterprise-scale operations:
Experiment: A logical grouping representing a use case (e.g., “fraud-detection-v2”). All runs exploring this problem space live here.
Run: An individual training execution with its own parameters, metrics, and artifacts. Each pipeline execution or manual training job maps to one run.
Run group: An optional intermediate grouping for related runs, useful for organizing hyperparameter sweeps or A/B comparisons.
Automatic and explicit tracking
When we launch a SageMaker training job with an experiment name, the service automatically captures the instance type, hyperparameters passed to the estimator, input data channel locations, and output artifact S3 paths. No instrumentation code is required for these baseline attributes.
For richer tracking, the Experiments SDK provides explicit logging within a run context: log_parameter() records hyperparameters, log_metric() captures training and validation metrics with step-based ...