Search⌘ K
AI Features
Home
Courses
AI for Backend Engineers

Storing AI Output Properly

Understand how to build durable records of AI output by storing raw responses alongside parsed results, model and prompt versions, and timestamps. This lesson helps backend engineers ensure traceability, debugging capability, and reliability when integrating AI outputs into production services.

We'll cover the following...

The previous lesson's route_feedback function runs and forgets. It reads a validated Feedback object, takes an action, and returns a string, and nothing about what actually happened is kept anywhere. That is fine for a demonstration. It is a real liability for a production feature, because the day someone asks why a ticket got created, or what the model actually said, there needs to be an answer sitting somewhere durable, not a shrug. This lesson builds that record.

Already configured setup code

The code below is carried over so this file runs entirely on its own: the schema a valid result must match, the prompt builder, the code fence stripper, a call_llm that runs with no API key using a mock response by default, and the pipeline tying it all together, plus the routing functions from the previous lesson. None of this is new. It exists here purely so this lesson does not depend on any other file.

import json
from enum import Enum
from pydantic import BaseModel, Field, ValidationError
class Sentiment(str, Enum):
    POSITIVE = "positive"
    NEUTRAL = "neutral"
    NEGATIVE = "negative"
class FeedbackCategory(str, Enum):
    PRODUCT_QUALITY = "product_quality"
    CUSTOMER_SERVICE = "customer_service"
    PRICING = "pricing"
    OTHER = "other"
class Confidence(str, Enum):
    HIGH = "high"
    LOW = "low"
class Feedback(BaseModel):
    sentiment: Sentiment
    category: FeedbackCategory
    severity: int = Field(ge=1, le=5, default=1)
    confidence: Confidence
    summary: str
def build_feedback_prompt(feedback_text: str) -> str:
    return (
        "Analyze the customer feedback below. Respond with a JSON object "
        "in exactly this shape, and nothing else:\n\n"
        "{\n"
        '  "sentiment": <one of: "positive", "neutral", "negative">,\n'
        '  "category": <one of: "product_quality", "customer_service", '
        '"pricing", "other">,\n'
        '  "severity": <integer from 1 to 5>,\n'
        '  "confidence": <one of: "high", "low">,\n'
        '  "summary": <string, one short sentence>\n'
        "}\n\n"
        f'Feedback: "{feedback_text}"'
    )
def build_repair_prompt(original_prompt: str, bad_response: str, error: ValidationError) -> str:
    return (
        f"{original_prompt}\n\nYour previous response did not match the "
        f"required format. Previous response:\n{bad_response}\n\n"
        f"Validation error:\n{error}\n\nRespond again with only the "
        "corrected JSON object."
    )
def strip_code_fence(text: str) -> str:
    stripped = text.strip()
    if stripped.startswith("```"):
        lines = stripped.split("\n")[1:]
        if lines and lines[-1].strip() == "```":
            lines = lines[:-1]
        return "\n".join(lines).strip()
    return stripped
# USE_MOCK is True by default, so this file runs immediately with no
# API key. Set it to False and fill in a real provider block below to
# use an actual model.
USE_MOCK = True
RAW_MOCK_RESPONSE = (
    '{"sentiment": "negative", "category": "customer_service", '
    '"severity": 4, "confidence": "high", '
    '"summary": "Support response took four days."}'
)
def call_llm(prompt: str) -> str:
    if USE_MOCK:
        return RAW_MOCK_RESPONSE
    # from openai import OpenAI
    # client = OpenAI(api_key="your-openai-api-key")
    # response = client.chat.completions.create(
    #     model="your-chosen-openai-model",
    #     messages=[{"role": "user", "content": prompt}],
    # )
    # return response.choices[0].message.content
    raise NotImplementedError("Fill in a real provider block, or set USE_MOCK back to True.")
def get_validated_feedback(feedback_text: str, max_attempts: int = 2) -> Feedback:
    prompt = build_feedback_prompt(feedback_text)
    current_prompt = prompt
    for attempt in range(1, max_attempts + 1):
        raw_response = call_llm(current_prompt)
        cleaned = strip_code_fence(raw_response)
        try:
            parsed = json.loads(cleaned)
        except json.JSONDecodeError:
            continue
        try:
            return Feedback.model_validate(parsed)
        except ValidationError as error:
            current_prompt = build_repair_prompt(prompt, cleaned, error)
    return Feedback(
        sentiment=Sentiment.NEUTRAL,
        category=FeedbackCategory.OTHER,
        severity=3,
        confidence=Confidence.LOW,
        summary="Could not be automatically assessed.",
    )
def create_ticket(feedback: Feedback) -> None:
    print(f"Creating ticket. Category: {feedback.category.value}. Summary: {feedback.summary}")
def notify_channel(feedback: Feedback) -> None:
    print(f"Notifying #feedback channel. Summary: {feedback.summary}")
def archive_silently(feedback: Feedback) -> None:
    print(f"Archived with no action. Category: {feedback.category.value}.")
def route_feedback(feedback: Feedback) -> str:
    if feedback.confidence == Confidence.LOW:
        notify_channel(feedback)
        return "notified_low_confidence"
    if feedback.severity >= 4:
        create_ticket(feedback)
        return "ticket_created"
    if feedback.severity >= 2:
        notify_channel(feedback)
        return "notified"
    archive_silently(feedback)
    return "archived"

Notice RAW_MOCK_RESPONSE is now pulled out as its own named constant, rather than sitting inline inside call_llm. That is deliberate, and it is the first hint of this lesson's actual topic: this lesson cares about the raw text itself as a thing worth keeping, not just the parsed result it eventually becomes.

A schema for the full record

This is what the lesson is actually about. The obvious first instinct when storing an ...