Handle Rate Limits Without Wasting Your Budget

A script that processes fifty study notes should not need constant supervision. Yet a burst of requests can leave you with partial results and a tempting but unhelpful response: restart everything. Instead, build a small recovery routine around each request. This walkthrough uses a note-summarizing script as an example, with practical limits on waiting, retries, and spending.

Start by identifying what actually stopped the request

When a request fails, inspect both the status and the error message before deciding to retry. A 429 response can indicate requests arriving too quickly, but an API may also use it for exhausted credits or a usage limit. Temporary throttling calls for slower traffic; a depleted balance calls for a decision about spending, not another attempt. ([developers.openai.com](https://developers.openai.com/api/docs/guides/error-codes?utm_source=openai))

Give your script separate outcomes for temporary throttling, account limits, and other failures. Record the affected note's identifier and a short explanation. Avoid copying private note contents or your API key into logs.

Put unfinished work into a controlled queue

Before adding recovery logic, make the workload manageable. Place your notes in a queue and begin with one request in flight. Add a deliberate gap between requests rather than launching the whole collection together. Treat this as your starting policy, not a claim about any provider's allowance.

Only increase concurrency after a small test finishes reliably. Keep a separate pacing rule for how often requests begin: limiting simultaneous work alone is not your complete traffic policy. Favor a steady trickle over finishing a few seconds earlier.

Let the response guide the waiting time

For a retryable failure, look for a Retry-After response header. When supplied, it tells your client how long to wait before trying again. Follow that instruction, provided the wait fits within your script's overall deadline. If it does not, save the unfinished item and stop cleanly instead of retrying early. ([help.openai.com](https://help.openai.com/en/articles/5955604-troubleshooting-api-rate-limits-and-429-errors?utm_source=openai))

Make waiting visible with a brief progress message. Knowing that the script is paused deliberately is more useful than watching an apparently frozen process and starting a second copy.

Use increasing delays when no waiting instruction arrives

When there is no usable waiting instruction, apply exponential backoff with jitter: increase the delay after each unsuccessful attempt and add randomness so retries do not all happen together. OpenAI recommends this approach for rate-limit recovery. ([developers.openai.com](https://developers.openai.com/api/docs/guides/rate-limits?utm_source=openai))

For your initial policy, you might allow three retries, begin around two seconds, and lengthen subsequent waits. Also set a total elapsed-time limit. These are adjustable starting choices, not universal settings. Once either limit is reached, return the item to your unfinished-work list.

Choose one place to control retries

Check your SDK before adding an outer retry loop. Official OpenAI SDKs already retry eligible failures and honor Retry-After when present. An additional retry layer can multiply attempts beyond what you intended. Decide which layer owns recovery, then make its limits explicit. ([help.openai.com](https://help.openai.com/en/articles/5955604-troubleshooting-api-rate-limits-and-429-errors?utm_source=openai))

For the note example, avoid automatically restarting the entire collection after one exhausted retry sequence. Keep successful summaries, mark the failed note as unfinished, and move on or pause according to your chosen policy.

Make recovery respect your spending limit

Assign the run a budget before processing the full queue. Track completed items and review actual usage after a small sample. With MyRelay, one personal key and a shared pay-as-you-go balance simplify access across providers, while per-request token and cost records help you inspect spending. Create a MyRelay account to review those records alongside your script's progress. ([myapirelay.com](https://myapirelay.com/))

Do not make automatic top-ups or model switches part of recovery by default. Require an explicit decision when the planned budget is reached, and preserve the remaining queue.

Test the pause before trusting the whole run

Simulate a temporary limit, a missing waiting header, and exhausted retries. Confirm that the script waits, stops within its deadline, and preserves completed work. Then restart from the unfinished queue. Your success criterion is simple: interruptions should produce a controlled pause, not lost progress or an open-ended spending commitment.