Fix 429 Too Many Requests Errors in Claude and OpenAI APIs

A 429 response means the server understood your request, accepted your credentials, and still refused to process it right now. It is a throttling signal, not an authentication or billing failure. Your key is fine, your payload is fine, and retrying the identical request in a tight loop is the one reaction guaranteed to make things worse.

The fix depends entirely on which limit you hit, and the same status code hides at least four different ones. This triage is protocol-agnostic: it applies equally to OpenAI-compatible, Anthropic, and Gemini endpoints.

Four Different Limits Hide Behind One Status Code

Exponential backoff solves the first three because they are per-minute or per-connection windows that clear on their own. It does nothing for the fourth: if a daily quota is exhausted, waiting thirty seconds just burns thirty seconds.

Read the Response Before You Retry

Most clients collapse the server response into a generic "rate limited" toast. Get the raw body and headers first:

curl -i https://zivv.pro/v1/chat/completions \
  -H "Authorization: Bearer sk-your-key" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-5","messages":[{"role":"user","content":"Reply OK"}]}'

Two things matter in the output:

• The error message. Providers usually say which window tripped, for example "tokens per minute" versus "requests per minute" versus "quota exceeded". That single word decides your fix.
• The `retry-after` header. If it is present, it is the server telling you exactly how long to wait. Honor it instead of guessing.

If the message mentions tokens and your prompts carry long documents or big conversation histories, you have a TPM problem even if your request count is tiny. Five requests per minute with 80k-token prompts is far more traffic than two hundred short ones.

Retry with Exponential Backoff and Jitter

This is the baseline every production caller should have. It honors retry-after when the server provides it, backs off exponentially when it does not, adds jitter so parallel workers do not retry in lockstep, and gives up after a bounded number of attempts:

import random
import time
import requests

def call_with_backoff(payload, max_retries=5):
    for attempt in range(max_retries):
        resp = requests.post(
            "https://zivv.pro/v1/chat/completions",
            headers={"Authorization": "Bearer sk-your-key"},
            json=payload,
            timeout=300,
        )
        if resp.status_code != 429:
            return resp
        retry_after = resp.headers.get("retry-after")
        if retry_after is not None:
            delay = float(retry_after)
        else:
            delay = min(2 ** attempt, 30) + random.uniform(0, 1)
        time.sleep(delay)
    raise RuntimeError("Still rate limited after retries")

Three rules that people skip:

1. Cap the delay. Unbounded exponential growth turns attempt eight into a seventeen-minute sleep.
2. Add jitter. Without it, fifty workers that failed together retry together and trip the limit again in perfect sync.
3. Retry in exactly one layer. If your SDK already retries 429 internally and you wrap it in your own retry loop, five times five is twenty-five hidden attempts per logical call.

Cap Concurrency at the Source

If a serial run works and a parallel run fails, you are hitting a concurrency limit, and backoff alone leaves you oscillating: workers fail, sleep, then all wake and fail again. Put a semaphore or worker pool in front of the API and keep in-flight requests below the limit. Start conservatively, around 3 to 5 concurrent requests for batch jobs, and raise it only while 429s stay at zero. A queue that runs slightly below the limit finishes sooner than a burst pattern that keeps tripping it.

Reduce Token Pressure

For TPM limits, the lever is request size, not request timing:

• Trim conversation history instead of resending the entire session every turn
• Split large documents and summarize per chunk rather than pasting everything into one prompt
• Route cheap steps to a lighter model, for example gemini-3.5-flash for extraction and classification, and reserve claude-opus-4-8 for the reasoning steps. Current model IDs are listed in Model Hub

Halving average prompt size doubles your effective throughput under the same TPM ceiling without touching a single limit.

When It Is a Quota, Not a Rate Limit

If nothing works until the next day or the next billing window, you are not being rate limited, you are out of allowance. This is common with official subscription tiers, and Claude Code users feel it hardest because a single refactor session can consume what looks like a full day of quota. Backoff cannot fix an empty bucket.

The structural fix is moving that workload onto pay-as-you-go capacity. Zivv fronts 100+ models behind one key with account-pool scheduling on the backend, so a single account ceiling stops being your ceiling, and Claude Code connects natively over the Anthropic protocol. The setup is two environment variables, covered in Claude Code keeps hitting rate limits and the error reference.

FAQ

Should every 429 be retried automatically? Retry rate-window 429s with backoff, yes. Do not blind-retry quota-exhaustion 429s: read the error message, and if it says quota, alert instead of sleeping.

Why do I get 429 with only a handful of requests per minute? Almost always TPM. A few requests with very long prompts or large max_tokens outweigh hundreds of short ones. Check prompt sizes before checking request counts.

Does streaming help with rate limits? No. Streaming changes how you receive tokens, not how many you consume. It improves perceived latency and timeout behavior, but a streamed request counts against RPM and TPM exactly like a non-streamed one.

Will creating more API keys raise my limits? Usually not. Most limits apply at the account level, so ten keys share the same windows. Extra keys are still worth having for isolation and per-key caps, just not as a throughput trick.

If rate limits are interrupting real work rather than the occasional burst, the fix is capacity, not cleverness. Create a Zivv key, point your existing client at it, and keep your retry logic as a safety net instead of a way of life.