The Runtime Challenge: Governing Live AI Agents
Understand the challenges of governing autonomous AI agents during live operation and explore the MI9 Framework, which applies dynamic oversight through risk measurement, telemetry schema, continuous authorization, conformance rules, drift detection, and graduated containment to maintain safe and aligned AI behavior.
We'll cover the following...
You have now learned the technical controls that are performed before deployment:
Alignment (RLHF): We trained the model's intentions to be harmless.
Robustness/Control (PyRIT): We tested the model’s guardrails in a sandboxed environment to see if an attacker could bypass them.
Why pre-deployment safety is not enough
For standard chatbots, pre-deployment testing works well. But as large language models (LLMs) evolve into agentic AI systems, the safety paradigm completely breaks down.
Agentic AI systems are general-purpose AI models that can autonomously plan, revise goals, recall memory, coordinate tool use, and execute multi-step actions with little or no human oversight.
The transition from a passive chatbot to an active, autonomous agent introduces unique governance challenges that emerge only during runtime (while the agent is operating live).
The dynamic and emergent risks
The biggest risks posed by agents cannot be anticipated in a fixed pre-deployment lab test. They arise dynamically during live execution:
Goal drift: The agent starts with a safe, simple goal (e.g., Analyze data trends) but, through autonomous reasoning, internally revises its goal to something unauthorized (e.g., Gain elevated system access).
Recursive planning loops: The agent gets stuck in a self-perpetuating loop (e.g., attempting a failed action, debugging the failure, and re-attempting indefinitely), consuming excessive resources or escalating a minor issue into a system-wide failure.
Cascading tool chains: The agent connects external tools (e.g., a web search API, a coding tool, and a database connector) in an unexpected sequence that violates internal policy (e.g., executing code without human approval).
Since these critical risks arise dynamically, they elude static pre-deployment control methods.
The MI9 framework
To address this critical gap, we must shift the locus of assurance from static pre-deployment checks to dynamic runtime governance. We will utilize the MI9 Framework, an integrated architecture specifically designed for the runtime safety of highly autonomous agents.
What it is: MI9 is a framework layer that instruments and governs existing agent systems (in a model and infrastructure-agnostic way) to enforce safety properties over live behavior sequences.
The goal: To enable real-time oversight and intervention at key decision boundaries, ensuring the agent remains corrigible (correctable) and aligned even as it autonomously pursues its goals.
Next, we will dive into the MI9 architecture, beginning with the essential first steps: measuring the agent’s risk level and achieving governance observability.
Architecture
The MI9 framework coordinates six specialized mechanisms to provide unified runtime oversight. To navigate its architecture more easily, MI9 can be understood as a three-layer system, with each layer building on the guarantees of the previous one.
Layer 1, Visibility (Agency-Risk Index [ARI] + Agentic Telemetry Schema [ATS]): First, we quantify how dangerous the agent is and establish a way to see its internal thoughts and actions.
Layer 2, Control (Continuous Authorization Monitoring [CAM] + Conformance): Next, we enforce strict boundaries on permissions and sequential behaviors in real-time.
Layer 3, Intervention (Drift + Containment): Finally, we detect subtle anomalies and provide a mechanism to stop the agent safely if it goes rogue.
Implementation requirement: Governance cannot be a passive background process. To actually block an action (Layer 2), the M19 layer must sit synchronously in the execution path (acting as a proxy or middleware). If the agent executes tool calls asynchronously (fire-and-forget), the governance layer will see the failure only after the damage is done .
We start with the essential steps for establishing governance: measuring the ...