overloaded_error (529): handling Claude API overloads

The short version

• overloaded_error is an HTTP 529: the Claude API is temporarily overloaded, which can happen under high traffic across all users.
• Your configuration is not the cause. Retry with backoff.
• While streaming, it can arrive as an error event after a 200, so a status check will not catch it.
• For overloads that last, fall back to another model.

What the error looks like

Anthropic's error docs define the shape: an error object with type and message, plus a request_id for tracing. A 529 looks like this (the ID is illustrative):

{
  "type": "error",
  "error": {
    "type": "overloaded_error",
    "message": "Overloaded"
  },
  "request_id": "req_011CSHoEeqs5C35K2UUqR7Fy"
}

529 versus 429

Anthropic also notes that a sharp increase in your own usage can produce 429s from acceleration limits, so ramp traffic up gradually.

Errors in the middle of a stream

A stream can fail after the 200. Anthropic's streaming docs show this event:

event: error
data: {"type": "error", "error": {"type": "overloaded_error", "message": "Overloaded"}}

You will already have partial output, so decide up front what your UI does with it.

Backoff and fallback

Anthropic's official SDK retries transient failures twice by default with exponential backoff (max_retries changes the count). If more attempts are all you need, set that and stop here.

This version turns SDK retries off, retries each model, then moves to the next, honoring retry-after when it is present:

import os
import random
import time

from openai import APIConnectionError, APIStatusError, OpenAI

client = OpenAI(
    api_key=os.environ["FASTMETAL_API_KEY"],
    base_url="https://api.fastmetal.ai/v1",
    max_retries=0,
)

RETRYABLE = {500, 502, 503, 504, 529}


def wait_seconds(err: Exception, attempt: int) -> float:
    if isinstance(err, APIStatusError):
        retry_after = err.response.headers.get("retry-after", "")
        if retry_after.isdigit():
            return float(retry_after)
    return min(30, 2**attempt) + random.random()


def create_with_fallback(models: list[str], messages: list[dict], attempts: int = 4):
    last_error: Exception | None = None
    for model in models:
        for attempt in range(attempts):
            try:
                return client.chat.completions.create(
                    model=model, messages=messages, max_tokens=1024
                )
            except APIStatusError as err:
                if err.status_code not in RETRYABLE:
                    raise
                last_error = err
            except APIConnectionError as err:
                last_error = err
            time.sleep(wait_seconds(last_error, attempt))
    raise last_error


resp = create_with_fallback(
    ["anthropic-claude-sonnet-5", "deepseek-v4.1-flash"],
    [{"role": "user", "content": "Hello"}],
)
print(resp.model, resp.choices[0].message.content)

A fallback model has its own output style and feature set. Test it against your prompts before you depend on it.

Fallback on FastMetal

FastMetal puts many models behind one API key, so the list above can span several labs with no extra integration work.

You can also leave fallback to the gateway. With model: "auto", FastMetal routes to the cheapest model that meets the request's needs and moves to the next candidate on failure. You pay the rate of the model that ran; see the pricing page.

FAQ

Am I charged for a request that returned overloaded_error?

Check the provider's billing terms for failed requests. Either way, cap your retries.

How many times should I retry?

There is no fixed number. Interactive UIs should give up quickly and fall back. Overnight batch jobs can afford to wait.

What should I check if 529s continue?

Look at the provider's status page for an incident. If only your organization is affected, consider whether a sudden traffic increase triggered limits.

Does OpenAI have the same error?

OpenAI's equivalent is a 503 with the code server_is_overloaded, documented in its error code guide.

Wrapping up

overloaded_error usually clears with time. Add backoff, then add a route to another model for the times it does not.

To use several models through one key, create a free account.