Design an Agentic Customer Support System

A single support bot looks attractive until it must issue refunds, update account data, inspect order state, and debug product failures while staying within strict latency, audit, and policy limits. The failure mode is predictable. One giant prompt accumulates too many tools, too much policy text, and too many action paths, so routing becomes fuzzy and risky actions become harder to bound.

This lesson designs the alternative. The system is an agentic support graph with a router, specialized domain agents, escalation supervision, and human handoff on one shared transcript. Requests move through controlled stages, not free-form autonomy.

The design target is operational, not cosmetic. The system should maximize resolution rate, keep turn latency low, prevent unsafe actions, and transfer context cleanly when automation stops. You will walk through requirements, estimation, architecture, subsystem design, workflow, and evaluation in the same order expected in a system design interview. That sequencing sets up the next lesson, where these explicit workflow controls tighten further into compliance-constrained generation.

System requirements

Before sizing anything, fix what the support system must do and how well it must do it. Every later section is checked against these requirements.

Functional requirements

• Resolve issues: Answer billing, order, and technical questions using retrieval plus account tools.

• Route by intent: Classify each request with a confidence score and ask once when unsure.

• Act safely: Execute refunds and account updates only through policy-checked tools.

• Verify identity: Require step-up verification before sensitive account changes.

• Escalate with context: Hand off to a human with a full handoff packet when automation stops.

Nonfunctional requirements

• Latency: Keep p95 time to first token under about 2 seconds.

• Scalability: Support 10,000 tickets per day, about 50,000 turns, and roughly 1,000 tickets per hour at peak.

• Availability: Degrade to informative mode when a tool is down, then escalate with state preserved.

• Reliability: Make refunds and account changes idempotent, with retries behind circuit breakers.

• Security and privacy: Use least-privilege credentials scoped to the authenticated account, and redact PII in logs.

• Cost: Bound tokens per ticket, about 21.5 million tokens per day at this load. ...

Resource estimation