LLM API 429 errors: rate limits, spent balances and the fix
The short version
- HTTP 429 means Too Many Requests. On an LLM API it usually means you passed a requests-per-minute or tokens-per-minute limit.
- On a prepaid gateway, the same status can also mean the balance is spent. On FastMetal the response body then contains
budget_exceeded. - Back off and retry a real rate limit. Never retry a spent balance, because waiting changes nothing.
- Read the response body before you decide. The status code alone cannot separate the two cases.
A retry loop that treats every 429 the same will spin forever on a key with no money left. The fix is a single branch in your error handler.
What a rate limit counts
Providers limit how much one key can send in a time window, usually on two axes:
| Limit | What it counts | When you hit it |
|---|---|---|
| RPM (requests per minute) | Number of requests | Many short requests in parallel |
| TPM (tokens per minute) | Tokens processed | Long prompts or long outputs in a row |
Requests over the limit are rejected at once. LLM inference runs on shared GPUs, so these limits stop one caller from slowing everyone else down.
The numbers vary by provider, model and contract, and they change. Design for a 429 arriving at any time, and keep specific numbers out of your logic.
The two meanings of 429
You sent too much
The RPM or TPM window is full. It refills with time, so a short wait and a resend will succeed.
Your prepaid balance is spent
On a prepaid API like FastMetal, the balance you bought is the spending cap. Once it is used up, requests are rejected with a 429 and budget_exceeded in the response body.
No amount of waiting fixes this. The request can only succeed after someone adds credit. FastMetal emails you when the balance runs low and again when it runs out, so a person usually hears about it before the code does.
How to handle each case
Exponential backoff with jitter
For a real rate limit, double the wait on each attempt: 1 second, 2, then 4. Add a random offset so that parallel clients do not all retry in the same instant. If the response carries a Retry-After header, use that value instead.
Cap the attempts, then return the error to the caller.
Stop immediately on a spent balance
When you detect budget_exceeded, raise an exception and move to whatever comes next in your product: a top-up prompt, an alert to an admin, a clear message to the user.
Limit concurrency
It is cheaper to avoid a 429 than to recover from one. A semaphore caps how many requests run at once. For batch jobs, put the work in a queue and drain it steadily.
Send fewer tokens
TPM limits count tokens, so shorter prompts and a tighter max_tokens let the same quota process more requests.
A Python example
This points the OpenAI SDK at FastMetal and handles the two cases separately:
import random
import time
import openai
from openai import OpenAI
client = OpenAI(
api_key="<FASTMETAL_API_KEY>",
base_url="https://api.fastmetal.ai/v1",
)
def chat(messages, max_retries=5):
for attempt in range(max_retries):
try:
return client.chat.completions.create(
model="anthropic-claude-haiku-4-5",
messages=messages,
max_tokens=300,
)
except openai.RateLimitError as e:
if "budget_exceeded" in str(e):
raise RuntimeError("Balance is spent. Add credit.") from e
wait = min(2 ** attempt + random.uniform(0, 1), 30)
time.sleep(wait)
raise RuntimeError("Gave up after repeated rate limits.")
resp = chat([{"role": "user", "content": "Hello"}])
print(resp.choices[0].message.content)
The branch inside except is what matters. To call this in parallel, wrap it in a threading.Semaphore or asyncio.Semaphore.
Free FastMetal models, whose ids end in -free, have their own per-key rate limits. A 429 on one of those takes the same backoff.
FAQ
Should I always wait and retry on a 429?
Only for a real rate limit. On a prepaid API a 429 can also mean the balance is spent. If the body contains budget_exceeded, retrying does nothing.
Where do I find the actual limit numbers?
They differ by provider and model, and they change, so check the provider's own limits page. Build backoff and concurrency control so that your code keeps working when the numbers move.
How many times should I retry?
Three to five attempts is a common cap. Unbounded retries pile up work during an outage and make the cause harder to find.
Wrapping up
Read the body of every 429. Back off on a rate limit, and stop on budget_exceeded.
The code above runs as written against FastMetal's OpenAI-compatible API. Per-model rates are on the pricing page, and you can create a free account to get a key.