Handle API Failures with Retry and Fallback

A support triage workflow may run normally for hours before some tickets fail to reach the triage step. Run history shows errors such as 429 Too Many Requests, request timeouts, and occasional 502 Bad Gateway responses. The downstream step that records category, priority, and next_action cannot proceed because no usable model response was returned.

This lesson turns that incident pattern into deterministic workflow behavior for transport and provider failures, separate from output validation.

When the request fails, no output exists

Looking at run history snapshots side by side makes the shared problem visible.

• Rate limited: HTTP 429 and an error body or message indicating rate limiting, with no usable triage JSON in the response field.

• Timed out: ETIMEDOUT or Request timed out, with an elapsed time near the configured timeout and no response body.

• Server error: HTTP 5xx like 500 or 502 with a provider error message, and no trustworthy response body.

In all three cases, the HTTP request did not succeed, so there is no triage output to validate against the schema.

The key separation is that error handling triggers on transport and provider failures, while output validation triggers only after a successful response exists. Validate category, priority, next_action, and needs_human only when the HTTP step returns success and a response body is present.

Rule: Validate outputs only after the workflow has a successful HTTP response.

Debug from evidence, not assumptions

With the failure types separated, debugging starts at the HTTP step that calls the LLM endpoint and stays grounded in what the run record captured. Status code, error text, and elapsed time form a complete first pass because they distinguish rate limiting from timeouts from server errors.

A practical routine keeps the workflow platform-agnostic while still being concrete:

• Status first: Check whether a status code exists, and record it as status.

• No status: If ...