Search⌘ K
AI Features
Home
Courses
AI Solutions Architect

Decision Ladder: Selecting the Minimum Sufficient Step

Explore the decision ladder framework to identify the minimum sufficient architectural step for AI systems. Understand how to balance KPI targets, privacy, latency, and cost constraints while avoiding over-building. Learn to define clear escalation triggers and evidence plans for moving between prompting, retrieval-augmented generation, tools, agents, and adaptation steps to maintain compliance and operational efficiency.

We'll cover the following...

The decision ladder is an architectural control against over-building. Start at the simplest step that can plausibly meet the KPI targets and feasibility constraints already qualified, and escalate only when a measured gap can be named, along with the evidence that will confirm it.

Feasibility constraints act as caps on how far up the ladder you’re allowed to go. Privacy constraints can prohibit certain data movements that later steps tend to assume. Latency and cost constraints can rule out architectures that introduce repeated loops, long context growth, or high retry rates. So the ladder selects two things at once: a minimum sufficient step and a maximum viable step. The space between them is the design area for the three-option solution paper.

The ladder has five steps:

  • Prompting only, with structured outputs

  • Retrieval-augmented generation (RAG)

  • Tools, for authoritative reads and writes

  • Agents, for dynamic multi-step planning

  • Adaptation, for behavior that changes with experience

The ...

Decision Ladder: Evidence Triggers and New Failure Modes
StepEvidence Trigger to EscalateNew Failure Modes Introduced

Prompting-Only/Structured Outputs

Repeatable missing or disputed policy facts that can't be supplied in the prompt without violating latency, privacy, or operational constraints.

Format drift: inconsistent structure, verbose answers, prompt sensitivity, brittle constraints.

RAG

Missing or stale knowledge: the model lacks required facts or domain context; hallucinations or guesswork observed.

Retrieval noise: irrelevant or low-quality documents, chunking issues, stale indexes, source misattribution.

Tools

Need for live data or actions: requires external system access, calculations, or real-world interactions.

Tool errors & side effects: wrong tool choice, bad inputs, API failures, rate limits, cost, security exposures.

Agents

Multi-step task failure: breaks down on planning, coordination, or long-horizon tasks.

Compounding agent mistakes: planning errors, wrong tool sequences, state drift, hard-to-debug emergent behaviors.

Adaptation

Repeated error patterns: systematic failures persist across contexts; the system needs to learn and improve.

Regression risk from adaptation: drift, overfitting, unintended behavior changes, eval leakage, data/feedback bias.

Do you find this helpful?
...