Search⌘ K
AI Features
Home
Courses
AI for Backend Engineers

When Validation Fails

Explore how to handle AI model output that fails schema validation by using a targeted repair loop. Learn to feed validation errors back to the model for corrections, limit retry attempts, and implement safe fallbacks. This lesson ensures your backend service remains reliable, never crashes, and always returns valid structured data despite AI unpredictability.

We'll cover the following...

The previous lesson ended with ValidationError being raised the moment a response didn't match our schema, and nothing after that point. That's honest, but it's not yet a complete system; a real service can't just stop the moment one response doesn't validate. This lesson builds what happens next: a repair loop that gives the model one or two genuine chances to fix its own mistake, using the actual validation error as the correction signal, before giving up in a defined, safe way.

Feeding the error back to the model

The single most useful piece of information we have after a failed validation is the error message itself; it names exactly which field was wrong and why. Instead of discarding that and just asking again with the same prompt, we can hand the error straight back to the model as part of a second attempt.

Python
from pydantic import BaseModel, Field, ValidationError
class TicketExtraction(BaseModel):
    severity: int = Field(ge=1, le=5)
    reason: str
def build_repair_prompt(original_prompt: str, bad_response: str, error: ValidationError) -> str:
    return (
        f"{original_prompt}\n\n"
        "Your previous response did not match the required format. "
        f"Your previous response was:\n{bad_response}\n\n"
        f"The validation error was:\n{error}\n\n"
        "Please respond again, correcting this specific problem, and "
        "return only the corrected JSON object."
    )
original_prompt = (
    'Respond with JSON: {"severity": <integer 1-5>, "reason": <string>}. '
    'Bug: "Search is broken for all users."'
)
bad_response = '{"severity": "very high", "reason": "Search is broken for all users."}'
try:
    TicketExtraction.model_validate_json(bad_response)
except ValidationError as error:
    repair_prompt = build_repair_prompt(original_prompt, bad_response, error)
    print(repair_prompt)

  • Lines 10–17: build_repair_prompt doesn't throw away the context of what went wrong, it includes the original instructions, the exact bad response the model produced, and the exact validation error, all in one follow-up prompt. This gives the model something far more specific to correct than a bare "try again" would.

    • Line 14: Passing error directly into the f-string relies on Pydantic's ValidationError having a clear, readable string ...