Search⌘ K
AI Features
Home
Courses
Applied AI Engineer: Build, Evaluate, and Operate Production LLM

Wrapping Models in APIs

Explore how to wrap large language models in robust FastAPI endpoints that validate inputs, separate prompt logic from transport, support streaming responses, and handle errors safely. This lesson guides you to design maintainable, testable API contracts essential for deploying reliable AI services.

We'll cover the following...

A stable API boundary turns an LLM pipeline into something other services can call without inheriting your internal prompt code. Start with the smallest FastAPI contract that validates input and returns a predictable JSON shape. Then widen the boundary to support streaming and failure mapping while keeping provider variability behind a service layer.

This lesson covers a single POST endpoint, a service function boundary, an event stream option, and an error model.

To ground the contract, examine the minimal FastAPI route below, which uses typed request and response models.

class CreateWidgetRequest(BaseModel):
    model_config = ConfigDict(extra="forbid")
    name: str = Field(min_length=1, max_length=60)
    quantity: int = Field(ge=1, le=1_000)
    note: str | None = Field(default=None, max_length=200)
    priority: int = Field(default=0, ge=0, le=10)
class WidgetResponse(BaseModel):
    model_config = ConfigDict(extra="ignore")
    id: str
    name: str
    quantity: int
    note: str | None
    priority: int
    created_at: datetime
app = FastAPI()
@app.post("/widgets", response_model=WidgetResponse)
def create_widget(payload: CreateWidgetRequest) -> WidgetResponse:
    return WidgetResponse(
        id=str(uuid4()),
        name=payload.name,
        quantity=payload.quantity,
        note=payload.note,
        priority=payload.priority,
        created_at=datetime.now(timezone.utc),
    )
A minimal typed FastAPI route: a request model, a response model, and the handler in between

  • Line 2 (model_config = ConfigDict(extra="forbid")): Any request body containing an unlisted field is rejected outright instead of having the extra field silently dropped.

  • Lines 4–7 (the Field(...) constraints): Pydantic enforces each constraint (min_length, ge/le, etc.) before create_widget runs, so a quantity of 0 or a name of "" never reaches the handler body.

  • Line 11 (model_config = ConfigDict(extra="ignore")): The opposite policy applies to responses: if the object being converted to WidgetResponse carries extra attributes, they're dropped from the JSON instead of causing an error.

  • Line 24 (@app.post("/widgets", response_model=WidgetResponse)): Declares the endpoint's guarantee: every successful response matches WidgetResponse exactly, and unlisted fields never leak into the JSON.

  • Lines 25–33 (create_widget): By the time this function's first line executes, payload is already validated, so the function only constructs the response and never checks its own input.

Here's what that buys you locally, with no HTTP request involved:

CreateWidgetRequest.model_validate({"name": "", "quantity": 0, "extra": "nope"})
# raises pydantic.ValidationError with three separate field failures:
# name too short, quantity below its minimum, and "extra" not permitted
Triggering the same validation FastAPI runs automatically

FastAPI validates the request body against the Pydantic model before your handler runs, so ...