Sensitivity Analysis and Breakpoint Discovery
Explore how to apply sensitivity analysis and breakpoint discovery to focus on the main factors affecting AI system cost, latency, and throughput. Understand how to assign ownership, model correlated parameter shifts, and translate breakpoints into architecture adjustments that comply with business constraints. This lesson equips you to create actionable decision artifacts and prepare evidence plans crucial for AI system governance and operational readiness.
Latency budgets and degraded modes give the system a bounded worst case. They don’t say which assumption in the underlying model is actually worth arguing about, versus which one barely moves the number. Sensitivity analysis exists to create decision leverage, not precision. Treat the unit economics model from the last lesson as a parameterized placeholder, and find which assumption moves cost_per_successful_task, p95_latency, and approval_throughput the most, so measurement, caps, and redesign effort can focus on the true driver.
Start from a baseline scenario anyway, since it sets a shared starting point and makes assumption ownership explicit. Give every input an owner and a planned replacement signal, so telemetry and evaluation results can overwrite the baseline rather than defend it as fact:
Product owns volume and quality thresholds, and defines what counts as a successful task and acceptable rework.
Platform owns model routing assumptions, context assembly policy, and the operational shape of retries and timeouts.
Security and governance own which tasks require human review and what evidence approval requires.
Operations owns incident assumptions, peak load multipliers, and the realism of tool error rates and recovery behavior.
Include at minimum daily_volume, avg_context_tokens, agent_step_count, fallback_rate, and human_review_rate in ...