context_length_exceeded: causes and a code fix for LLM APIs

The short version

• context_length_exceeded is a 400: the request does not fit the model's context window. Sending it again will fail the same way.
• Two things cause it. Either the input is too large, or max_tokens reserves so much output that input plus reservation goes over.
• Anthropic reports the same condition as a 400 with the message prompt is too long, so a check on code alone misses it.
• The fix is one of four: lower max_tokens, drop old history, summarize it, or move to a model with a bigger window.

What the error looks like

OpenAI's Chat Completions API returns this body with HTTP 400. This one comes from a report on OpenAI's developer forum:

{
  "error": {
    "message": "This model's maximum context length is 8192 tokens. However, you requested 9850 tokens (1750 in the messages, 8100 in the completion). Please reduce the length of the messages or completion.",
    "type": "invalid_request_error",
    "param": "messages",
    "code": "context_length_exceeded"
  }
}

Read the numbers in parentheses first. The input here is only 1,750 tokens, and the 8,100 reserved by max_tokens is what breaks the limit. Check max_tokens before you cut any input.

OpenAI's docs list the same code for input that exceeds the model's context window. On the Responses API, developers have reported a different 400 message: Your input exceeds the context window of this model. Please adjust your input and try again.

Anthropic: prompt is too long

Anthropic's docs say input that alone exceeds the window returns a 400 invalid_request_error with the message prompt is too long. Anthropic's error object has type and message, and no code.

Claude 4.5 and newer behave differently when only input plus max_tokens goes over. The API accepts the request, and if generation reaches the edge of the window it stops with stop_reason: "model_context_window_exceeded". You get a truncated answer and no exception, so check the stop reason as well.

Finding the cause

To estimate before sending, read usage from earlier responses or call FastMetal's count tokens endpoint on the Anthropic-compatible API.

Trim the history and retry

The wording differs by API, so test both the code and the message. When the request overflows, this drops the older half of the conversation and makes sure it still starts on a user turn. A leading orphaned tool result would trigger a different error.

import os

from openai import BadRequestError, OpenAI

client = OpenAI(
    api_key=os.environ["FASTMETAL_API_KEY"],
    base_url="https://api.fastmetal.ai/v1",
)

OVERFLOW_MARKERS = (
    "maximum context length",
    "exceeds the context window",
    "prompt is too long",
)


def is_context_overflow(err: BadRequestError) -> bool:
    if err.code == "context_length_exceeded":
        return True
    return any(marker in err.message.lower() for marker in OVERFLOW_MARKERS)


def complete(system: dict, turns: list[dict], max_tokens: int = 1024):
    while True:
        try:
            return client.chat.completions.create(
                model="deepseek-v4.1-flash",
                messages=[system] + turns,
                max_tokens=max_tokens,
            )
        except BadRequestError as err:
            if not is_context_overflow(err) or len(turns) <= 1:
                raise
            turns = turns[len(turns) // 2:]
            while len(turns) > 1 and turns[0]["role"] != "user":
                turns = turns[1:]

Trimming loses decisions and numbers from earlier in the conversation. For long sessions, replace old turns with a summary that keeps those facts.

Switching to a bigger window on FastMetal

FastMetal exposes many models behind one OpenAI-compatible endpoint, so a bigger window is a change to the model string. deepseek-v4.1-flash handles a 1M-token context, and the catalog shows the window for every other model.

Billing is per million tokens, and every token you send counts as input however large the window is. Current rates are on the pricing page.

FAQ

Will retrying fix context_length_exceeded?

No. The request is the same size every time. Exclude this error from your retry logic and change the input or max_tokens.

Does leaving out max_tokens prevent it?

Only when the reservation was the problem. If the input alone is over the limit, the request still fails.

My output is cut off but there is no error. Is that the same problem?

It is a related one. A response that stops at the limit returns HTTP 200 with finish_reason: "length", so check that field on every response.

Is it safe to fill the window right up to the limit?

Accuracy tends to drop as the context fills, even inside the limit. Send what the task needs.

Wrapping up

Treat context_length_exceeded as a request to fix, never one to retry. The numbers in the message tell you whether input or max_tokens is responsible.

To try models with larger windows through one key, create a free account.