Search⌘ K
AI Features
Home
Courses
Applied AI Engineer: Build, Evaluate, and Operate Production LLM

Wrapping Models in APIs

Explore how to wrap LLM pipelines into reliable APIs by establishing clear request and response contracts using FastAPI. Learn to separate prompt logic from HTTP routes, enable streaming token output for improved client experience, and implement error handling models that ensure informative and secure failure responses. This lesson equips you to design stable, testable, and maintainable API services for LLM applications.

We'll cover the following...

A stable API boundary turns an LLM pipeline into something other services can call without inheriting our internal prompt code. Start with the smallest FastAPI contract that validates input and returns a predictable JSON shape, then widen the boundary to support streaming and failure mapping while keeping provider variability behind a service layer.

This lesson covers a single POST endpoint, a service function boundary, an event stream option, and an error model.

To ground the contract, examine a minimal FastAPI route below that uses typed request and response models.

class CreateWidgetRequest(BaseModel):
    model_config = ConfigDict(extra="forbid")
    name: str = Field(min_length=1, max_length=60)
    quantity: int = Field(ge=1, le=1_000)
    note: str | None = Field(default=None, max_length=200)
    priority: int = Field(default=0, ge=0, le=10)
class WidgetResponse(BaseModel):
    model_config = ConfigDict(extra="ignore")
    id: str
    name: str
    quantity: int
    note: str | None
    priority: int
    created_at: datetime
app = FastAPI()
@app.post("/widgets", response_model=WidgetResponse)
def create_widget(payload: CreateWidgetRequest) -> WidgetResponse:
    return WidgetResponse(
        id=str(uuid4()),
        name=payload.name,
        quantity=payload.quantity,
        note=payload.note,
        priority=payload.priority,
        created_at=datetime.now(timezone.utc),
    )
A minimal typed FastAPI route: A request model, a response model, and the handler in between

  • Line 2 (model_config = ConfigDict(extra="forbid")): A request body with any field not listed below is rejected outright, rather than the extra field being silently dropped.

  • Lines 4-7 (the Field(...) constraints): Each constraint (min_length, ge/le, etc.) is enforced by Pydantic before create_widget ever runs, a quantity of 0 or a name of "" never reaches the handler body.

  • Line 11 (model_config = ConfigDict(extra="ignore")): The opposite policy on the way out, if the object being converted to WidgetResponse happens to carry extra attributes, they're dropped from the JSON rather than causing an error.

  • Line 24 (@app.post("/widgets", response_model=WidgetResponse)): Declares the endpoint's guarantee: every successful response is shaped exactly like WidgetResponse, with unlisted fields never leaking into the JSON.

  • Lines 25-33 (create_widget): By the time this function's first line executes, payload is already validated, the function only has to construct the response, not check its own input.

What that buys you locally, with no HTTP request involved.

CreateWidgetRequest.model_validate({"name": "", "quantity": 0, "extra": "nope"})
# raises pydantic.ValidationError with three separate field failures —
# name too short, quantity below its minimum, and "extra" not permitted
Triggering the same validation FastAPI runs automatically

The key behavior is that FastAPI validates the ...