Define Versioned API Contracts for an LLM Endpoint
Explore how to define stable, versioned API contracts for LLM endpoints that separate unpredictable model outputs from operational metadata. Learn to use Pydantic models in FastAPI to enforce request and response schema validation, manage contract versions, and maintain compatibility. Understand version boundaries and best practices for safe API evolution to deliver reliable, structured LLM responses.
A Support Ticket Assistant caller expects stable JSON, but LLMs can produce variable, unstructured output. In the reference service used throughout this course, the FastAPI POST /support/assistant handler accepts a customer ticket payload and returns three top-level fields: draft_reply, rationale, and open_questions. Right now, the response is mocked. The integration constraints are already visible because the API contract must remain stable whether the response is mocked or generated by a model.
A typical downstream parser looks simple until the first rename lands. If the server changes open_questions to missing_info, a TypeScript client that reads resp.open_questions.length starts throwing at runtime. A Python client that does data["open_questions"] raises a KeyError. Nothing in the current API shape tells callers how to negotiate the change, and nothing in the current response tells observability tools which schema was intended.
Errors are worse because they surface as ambiguous strings instead of typed failures. A response like {"error": "model failed"} forces every caller to branch on free text. This makes retries, alerting, and dashboards inconsistent across teams.
The gateway owns a stable HTTP contract even if the provider API, prompt, or internal implementation changes underneath it. That contract needs explicit versioning and explicit rules for what can change without breaking callers.
Choose the contract surface area
With the boundary firmly established at the API Gateway, we can define exactly what the endpoint promises in requests and responses. Keep the request focused on the minimum the gateway needs to do to support triage.
A concrete request schema guarantees predictable inputs:
...