Search⌘ K
AI Features
Home
Courses
Applied AI Engineer: Build, Evaluate, and Operate Production LLM

Structured Outputs and Function Calling

Explore how to define and enforce strict JSON schemas and function calling in AI responses to ensure reliable, validated outputs. Understand error handling, schema versioning, and repair prompts to maintain production-grade data integrity and operational stability.

We'll cover the following...

Free-form text is a fragile interface when another program consumes the result. A trailing sentence or missing quotation mark can cause a JSON parse error. An unexpected key or incorrect value may still parse but fail validation or lead to misclassification.

This extraction task defines an output contract. Each support ticket maps to category, urgency, and summary. The response must be valid JSON that matches the required schema, with no surrounding prose, so downstream code can validate and use it.

Broken JSON fails, JSON-only parses successfully
Broken JSON fails, JSON-only parses successfully

The breakage pattern to watch is output that looks close to JSON but violates a strict parser. If downstream code runs JSON.parse and the response includes commentary after the closing brace, parsing fails immediately.

Rule to apply

If another program consumes the output, constrain the response with an explicit schema, not a vibe.

Enforce JSON-only outputs with a schema

Once the contract exists, the next problem is enforcement. The prompt should state the exact keys, forbid additional keys, and require strict types so the parser can reject ambiguous values like “urgent”: “high-ish” or urgency: 3 when an enum was intended.

A practical prompt template keeps the schema and the task text separated so logging and versioning stay simple.

System:
Return only valid JSON that matches the schema exactly.
No extra keys. No comments. No markdown.
User:
Task:
{{ticket_text}}
Schema:
{
  "type": "object",
  "additionalProperties": false,
  "properties": {
    "category": {"type": "string"},
    "urgency": {"type": "string", "enum": ["low","medium","high"]},
    "summary": {"type": "string"}
  },
  "required": ["category","urgency","summary"]
}
Prompt template

  • Line 1 (System:): Marks the start of the fixed instruction block, separate from the variable task text.

  • Lines 2–3: Tell the model its entire job is shape-compliance: no prose, no markdown fences, nothing after the JSON.

  • Line 5 (User:): Marks the start of the per-call, variable part of the prompt.

  • Lines 6–7 (Task: {{ticket_text}}): The only thing that changes between calls; everything else stays fixed, which is what makes versioning and logging simple.

  • Line 9 (Schema:): Labels the block that follows as the machine-readable contract, not more instructions.

  • Line 11 ("type": "object"): The top-level response must be a single JSON object, not an array or a bare string/number.

  • Line 12 ("additionalProperties": false): Rejects any key the model invents beyond the three listed; this is what stops silent schema drift.

  • Line 13 ("properties": {): Opens the block that defines each allowed field.

  • Line 14 ("category": {"type": "string"}): Free-text category for now (tightened to an enum later in the lesson).

  • Line 15 ("urgency": {"type": "string", "enum": [...]}): Restricts urgency to exactly three literal values, so the model cannot return "high-ish" or a number.

  • Line 16 ("summary": {"type": "string"}): Plain string, no further constraint needed since downstream code just displays it.

  • Line 18 ("required": [...]): All three fields must be present; a response missing summary fails validation immediately instead of becoming a null downstream.

Most providers expose a place in the API call to say return structured JSON and validate against a schema, but the parameter names differ. Treat the call shape as a pattern and verify the exact fields in our SDK.

OpenAI-style, structured output via response_format:

{
  "model": "<model-name>",
  "input": ["<system message>", "<user message>"],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "ticket_output",
      "strict": true,
      "schema": "<the schema above>"
    }
  }
}
OpenAI-style

  • Line 2 ("model"): Specifies which model handles the request; it does not define the schema or how it is enforced.

  • Line 3 ("input"): The system + user messages, same two-part prompt shown earlier.

  • Line 4 ("response_format": {): Opens the block that is the actual enforcement switch; it tells the API "don't just ask nicely, constrain generation to this schema."

  • Line 5 ( ...