Managing Retries
Explore how to handle retries in API calls during request failures. Understand different failure scenarios and the impact of excessive retries. Learn techniques like exponential backoff with jitter and retry capping to minimize server strain and network congestion.
Introduction
Digital applications receive millions of requests every second, but they might not complete their request-response cycle due to many possible failures. In this case, the client can generally receive the following HTTP status codes:
HTTP Status Code | Reason |
408 | Request timeout |
429 | Too many requests |
500 | Internal server error |
502 | Gateway or proxy server receives unexpected response |
503 | Service temporarily unavailable |
504 | Gateway or proxy server timeout |
Although each status code has its own story, we organized them into the following four categories. In the first two categories, the client doesn’t get a response from the server and needs to retry. In the last two categories, we receive the response with an error and need to retry after some modifications:
Request lost: The first scenario is when a request is initiated by the client but never received by the target service. It’s as if the request never occurred.
In the scenario above, the client doesn’t get a response. For example, requests can get lost due to congestion at some router while en route to the service.
Response lost: The second scenario is where the client initiates the request, which is processed by the target service. After this, the service sends the response to the requested client, which never arrives.
Service unresponsive: The third scenario is when the target service can’t process the request appropriately, possibly because of an error on the downstream service. The error status and the appropriate message are sent back to the client.
Response with error code: The fourth scenario can be where the client receives a response with an error from the target service due to a bad request or parameters. In this case, the client must address this error before retrying.
In the first two scenarios ...