How to Fix context_length_exceeded (OpenAI and Anthropic)

Figures on this page were verified 31 August 2026 against the providers' own documentation. Pricing, context windows and rate limits change without notice, so confirm any number against the provider before you rely on it. Tell us if something here is out of date.

Your request failed because the prompt plus the reply you asked for is larger than the model’s context window. The check is input_tokens + max_tokens > context_limit, and the second half is the part people forget: reserving 32,000 output tokens consumes 32,000 tokens of the window before the model writes a single word. Send less, ask for less, or move to a model with a larger window.

What the error looks like

Both providers return HTTP 400. The wording differs, which matters if you are matching on it.

OpenAI
  "code":    "context_length_exceeded"
  "message": "This model's maximum context length is N tokens.
              However, you requested M tokens ..."

Anthropic
  "type":    "invalid_request_error"
  "message": "prompt is too long: N tokens > M maximum"

Match on OpenAI’s code field rather than the message string. Message text changes between releases; the code does not.

The arithmetic that governs it

A context window is shared between everything you send and everything the model produces:

  system prompt
+ tool definitions
+ every prior turn in the conversation
+ this turn's message
+ max_tokens reserved for the reply
------------------------------------------
= must stay under the model's context limit

Two of those lines cause most failures. Prior turns are resent in full on every request, so a long conversation grows its own input until it hits the ceiling. Tool definitions are counted too, and a large tool schema can quietly occupy thousands of tokens on every single call.

Fix it in the order that costs you least

1. Lower max_tokens

The fastest fix, and the one people skip. If your prompt is 900,000 tokens and you reserved 128,000 for output on a 1M model, you are over the limit by 28,000 tokens before anything happens. Reserve what the reply actually needs, not the maximum the model allows.

2. Measure before you send

Counting locally turns a failed billable request into a branch in your code. Anthropic exposes a dedicated endpoint, which is exact because it uses the same tokenizer the model does:

from anthropic import Anthropic

client = Anthropic()

count = client.messages.count_tokens(
    model="claude-opus-5",
    messages=[{"role": "user", "content": long_text}],
)

print(count.input_tokens)

For OpenAI, count with tiktoken using the encoding for your model. Note that a tokenizer library is an approximation of the request as a whole: it counts your text, not the message envelope, tool schemas, or images, so leave headroom.

Do not use tiktoken to estimate Claude requests. It is OpenAI’s tokenizer and the token counts will not match.

3. Stop resending the whole conversation

In a chat loop you are not paying for one message, you are resending the entire transcript every turn. Keep a rolling window of recent turns, summarise older ones into a short running brief, and drop tool results you no longer need. If you are doing retrieval, pass the top matching chunks rather than the whole document.

4. Chunk, then combine

When the input genuinely is enormous, split it, process each piece, then run a second pass over the results. This turns one impossible request into several possible ones and is usually cheaper than reaching for a bigger model.

5. Move to a larger window, last

It is the obvious fix and the most expensive. A larger window does not make a bloated prompt correct, and you pay for every token you send on every call. Try the four steps above first.

Current context limits

Every figure below was read from the provider’s own documentation on the date shown, not from a secondary source.

Google does not publish token limits on its Gemini models overview page; they sit on each individual model’s page in the Gemini API documentation. Rather than repeat a number we could not verify, we have left Gemini out of the table.

A caveat on “1M tokens”

Tokenizers differ between providers and between model generations, so the same text does not produce the same token count everywhere. Treat any words-per-token rule as a rough estimate and measure the real thing when a request is anywhere near the ceiling. A model with a nominally larger window is not automatically the one that fits your prompt.

How to stop it happening again

• Count tokens before sending and branch, rather than catching a 400 after you have paid for it.
• Set max_tokens from the job at hand, not from the model maximum.
• Cap conversation history explicitly instead of letting it grow without limit.
• Audit tool schemas. They are sent on every request and are easy to forget.
• Handle the error as a distinct branch. It is not retryable: the identical request will fail identically, so retrying wastes time and money.

That last point is worth stating plainly. Generic retry logic treats every 400 the same and will loop on this one forever. Match the error code and shrink the request before trying again.

Related

Working out what a request costs before you send it is the same arithmetic viewed from the billing side. Our LLM API cost calculator prices a given token workload across every major model, and shows why output tokens dominate most bills. Our context budget planner shows how much of the window your fixed overhead consumes and at which turn a conversation breaks.

This page is part of our LLM API error reference, which covers every common error across OpenAI and Anthropic and whether each one is safe to retry.

Frequently asked questions

What actually counts toward the context window?

Everything you send plus everything the model may return: the system prompt, every prior turn, tool definitions, tool results, attached file text, and the max_tokens you reserve for the reply. Reserved output tokens are subtracted before a single token is generated.

Why do I get this error when my prompt looks small?

Usually max_tokens. Reserving 64,000 output tokens on a 200,000-token model leaves roughly 136,000 for input no matter how short the prompt feels. Conversation history and tool definitions then accumulate silently on top of that.

Is it better to truncate the start or the middle of a conversation?

The middle. Keep the system prompt and the most recent turns, and summarise or drop what is between them. Dropping the system prompt changes the model's behaviour, and dropping recent turns breaks coherence in a way users notice immediately.

How should I count tokens before sending?

Use the provider's own tokenizer: tiktoken for OpenAI, the count_tokens endpoint for Anthropic. The rule of thumb of about four characters per token is fine for planning a budget but not for staying under a hard limit.