Drift Detection and Investigation Workflow
Explore how to detect, scope, and investigate drift in AI systems that causes persistent quality changes. Understand the types of drift—data, prompt, and model version—and how to use telemetry, change logs, and offline evaluation for precise diagnosis. Learn a structured workflow to validate root causes, apply mitigations, and monitor recovery, ensuring reliable AI service operation within quality service level objectives.
We'll cover the following...
Reliability controls handle dependencies that fail outright. They do not address gradual quality degradation when the system remains operational. Drift is an operational condition in which observed behavior changes by an amount that is statistically and practically significant for a quality SLO and persists beyond the expected variation established by the baseline. Drift differs from expected random variation because it is persistent and may coincide with changes in the system, data, or user population. It requires monitoring that detects the shift and helps trace it to likely change events or affected components, such as a deployment, prompt-template update, index rebuild, routing-policy change, or tool-integration change.
The stakeholder decision in this lesson is which types of drift must be detected early to prevent a ...