Search⌘ K
AI Features
Home
Courses
Applied AI Engineer: Build, Evaluate, and Operate Production LLM

Structured Outputs and Function Calling

Explore the importance of structured outputs and function calling in prompt engineering to create machine-consumable JSON responses. Understand how to define strict schemas, validate outputs, and implement repair loops to handle errors. This lesson helps you build resilient AI systems by enforcing clear data contracts and integrating tool calls for bounded actions.

We'll cover the following...

Free form text is a fragile interface when another program needs to consume the result. A single trailing sentence, a missing quote, or an unexpected key can turn a valid looking answer into a parse error or worse into silent misclassification.

This small extraction task set a hard output contract. A support ticket can be mapped into category, urgency, and summary, but only if the output is always machine consumable JSON, not prose that happens to contain JSON sometimes.

Broken JSON fails, JSON-only parses successfully
Broken JSON fails, JSON-only parses successfully

The breakage pattern to watch is output that looks close to JSON but violates a strict parser. If downstream code runs JSON.parse and the response includes commentary after the closing brace, parsing fails immediately.

Rule to apply

If another program consumes the output, constrain the response with an explicit schema, not a vibe.

Enforce JSON-only outputs with a schema

Once the contract exists, the next problem is enforcement. The prompt should state the exact keys, forbid additional keys, and require strict types so the parser can reject ambiguous values like “urgent”: “high-ish” or urgency: 3 when an enum was intended.

A practical prompt template keeps the schema and the task text separated so logging and versioning stay simple.

System:
Return only valid JSON that matches the schema exactly.
No extra keys. No comments. No markdown.
User:
Task:
{{ticket_text}}
Schema:
{
  "type": "object",
  "additionalProperties": false,
  "properties": {
    "category": {"type": "string"},
    "urgency": {"type": "string", "enum": ["low","medium","high"]},
    "summary": {"type": "string"}
  },
  "required": ["category","urgency","summary"]
}
Prompt template

  • Line 1 (System:): Marks the start of the fixed instruction block, separate from the variable task text.

  • Lines 2-3: Tell the model its entire job is shape-compliance — no prose, no markdown fences, nothing after the JSON.

  • Line 5 (User:): Marks the start of the per-call, variable part of the prompt.

  • Lines 6-7 (Task: {{ticket_text}}): The only thing that changes between calls; everything else stays fixed, which is what makes versioning and logging simple.

  • Line 9 (Schema:): Labels the block that follows as the machine-readable contract, not more instructions.

  • Line 11 ("type": "object"): The top-level response must be a single JSON object, not an array or a bare string/number.

  • Line 12 ("additionalProperties": false): Rejects any key the model invents beyond the three listed — this is what stops silent schema drift.

  • Line 13 ("properties": {): Opens the block that defines each allowed field.

  • Line 14 ("category": {"type": "string"}): Free-text category for now (tightened to an enum later in the lesson).

  • Line 15 ("urgency": {"type": "string", "enum": [...]}): Restricts urgency to exactly three literal values — the model cannot return "high-ish" or a number.

  • Line 16 ("summary": {"type": "string"}): Plain string, no further constraint needed since downstream code just displays it.

  • Line 18 ("required": [...]): All three fields must be present — a response missing summary fails validation immediately instead of becoming a null downstream.

Most providers expose a place in the API call to say return structured JSON and validate against a schema, but the parameter names differ. Treat the call shape as a pattern and verify the exact fields in our SDK.

OpenAI-style, structured output via response_format:

{
  "model": "<model-name>",
  "input": ["<system message>", "<user message>"],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "ticket_output",
      "strict": true,
      "schema": "<the schema above>"
    }
  }
}
OpenAI-style

  • Line 2 ("model"): Which model handles the request — unrelated to the schema mechanism itself.

  • Line 3 ("input"): The system + user messages, same two-part prompt shown earlier.

  • Line 4 ("response_format": {): Opens the block that is the actual enforcement switch — it tells the API "don't just ask nicely, constrain generation to this schema."

  • Line 5 ("type": "json_schema"): Selects schema-constrained generation as the response mode.

  • Line 6 ("json_schema": {): Opens the object describing which schema to enforce.

  • Line 7 ("name": "ticket_output"): A label for this schema, ...