Timeouts, Retries, and UI-Level Rate Limiting
Explore how to manage and classify different timeout types implement retry logic without duplicating tool actions and apply UI-level rate limiting to prevent request floods. Understand how these guardrails keep AI feature responses reliable responsive and observable while preserving user control.
A streaming response can fail without throwing an error. The request connects, a few chunks arrive, and then the stream never reaches a final .done event, so the UI stays stuck in a generating state, and the "Stop" button becomes the only lifeline.
This lesson adds guardrails in the UI and at the proxy boundary so one stalled stream cannot wait forever, retries do not multiply side effects, and rapid user actions cannot flood the provider.
When a stream stalls, cap the wait
A stalled stream creates two problems at once. The UI cannot confidently conclude anything is wrong because no explicit error arrived, and the user cannot continue because the interface treats the turn as in flight.
The guardrail goals are concrete.
Cap worst-case wait time so the UI can recover to an actionable state.
Avoid duplicate side effects so retries do not create double tool calls or duplicate writes.
Keep user controls responsive so "Stop" always works and produces a clear outcome event. ... ...