Handle OpenAI API rate limits and transient errors robustly: read the rate-limit headers, retry only the right status codes with backoff and jitter, and understand tier limits

Documented steps

1. Read the rate-limit headers on every response: x-ratelimit-limit-requests, x-ratelimit-remaining-requests, x-ratelimit-reset-requests, x-ratelimit-limit-tokens, x-ratelimit-remaining-tokens, and x-ratelimit-reset-tokens.
2. Also handle the project-scoped variants where present: x-ratelimit-limit-project-tokens, x-ratelimit-remaining-project-tokens, and x-ratelimit-reset-project-tokens.
3. On a 429, check for the Retry-After header, which gives the minimum number of seconds to wait before retrying a temporary rate-limit error.
4. Retry ONLY 429s caused by temporary rate limiting. Do not retry quota, billing, or other errors that require developer action.
5. Treat Retry-After as a floor: wait at least that long, then add a small random jitter so concurrent clients do not retry in lockstep.
6. If Retry-After is missing or invalid, fall back to exponential backoff with jitter.
7. Cap both the retry attempt count and the total time spent retrying, then surface a clear error rather than retrying indefinitely.
8. Account for the official SDKs already retrying eligible rate-limit errors and honoring Retry-After. Do not stack a second retry loop on top without adjusting max_retries.
9. Model your limits correctly: they are enforced per organization AND per project (not per user), vary by model, and are measured by RPM, RPD, TPM, TPD, IPM, and audio minutes per minute.
10. Treat Batch API limits separately: they are computed on total input tokens queued per model and released when a batch job completes.
11. Check your organization's live limits at https://platform.openai.com/settings/organization/limits.
12. Plan for tier changes: organizations graduate to higher usage tiers automatically as cumulative paid spend increases, which generally raises limits across most models.