Search⌘ K
AI Features
Home
Courses
AI for Backend Engineers

Reading the Response

Understand how to process AI model responses beyond extracted text by exploring metadata like stop reasons and token usage. Learn to validate generated outputs to ensure completeness, handle truncation, and track usage accurately. This lesson builds essential safeguards for reliable backend AI integration.

We'll cover the following...

We spent the last lesson on the request side of the call, building the list of role-tagged messages sent to the model. Now we look at the response. It’s tempting to treat the response as only a text value, extract the generated text, and move on. That approach can fail because the generated text is only one part of the response. The remaining fields provide metadata we need to interpret, validate, and handle the response correctly.

A response is a structured object, not a bare string

Every provider wraps the generated text inside a larger response object that also carries metadata about the call: why the model stopped generating, and how many tokens the request and response actually used. The exact field names differ from provider to provider; we'll flag that clearly as we go, but the three pieces of information are present, in some form, in every response we'll get back.

Let’s simulate one so we can work with it directly, without needing a live API key yet.

Python
# A stand-in for a real response object. Field names vary by provider —
# check your chosen SDK's documentation for its exact response shape —
# but every provider's response carries some version of these three things.
response = {
    "generated_text": "The customer was charged twice due to a billing retry bug.",
    "stop_reason": "complete",
    "usage": {
        "input_tokens": 42,
        "output_tokens": 14,
    },
}
text = response["generated_text"]
print("Generated text:", text)

  • Lines 1–3: The comments explain that this dictionary simulates a model provider’s response. Real field names and available metadata vary by provider, model, and SDK.

  • Lines 4–11: We define the complete simulated response as a nested Python dictionary.

    • Line 5: generated_text contains the text produced by the simulated model ...