Figures on this page were verified 31 August 2026 against the providers' own documentation. Pricing, context windows and rate limits change without notice, so confirm any number against the provider before you rely on it. Tell us if something here is out of date.
Four things drive an LLM bill, and the price per token is the least interesting of them. Resent conversation history, output tokens costing several times more than input, retries, and cache misses explain almost every invoice that lands higher than the estimate. Model choice matters, but it is usually the smallest of the four levers.
1. You resend the whole conversation every turn
These APIs are stateless. The model has no memory of turn three when you send turn four, so your client resends everything: system prompt, tool definitions and every prior message. A twenty-turn conversation does not cost twenty messages. It costs the sum of a growing prefix.
With a 2,000-token system prompt and 300-token turns, turn one sends 2,300 tokens and turn twenty sends about 8,000. Across the conversation you have sent roughly 100,000 input tokens to exchange 6,000 tokens of actual content. That ratio, not the sticker price, is what makes chat expensive.
2. Output costs three to six times more than input
Every provider charges more for tokens it generates than for tokens you supply.
| Model | Input $/M | Output $/M | Ratio |
|---|---|---|---|
| Claude Opus 5 | $5 | $25 | 5.0x |
| Claude Sonnet 5 | $2 | $10 | 5.0x |
| GPT-5.6 Sol | $4 | $20 | 5.0x |
| GPT-5.6 Luna | $0.20 | $1.20 | 6.0x |
| Gemini 3.5 Flash | $1.50 | $9 | 6.0x |
The practical consequence is that verbosity is the most expensive habit in your prompt. Asking for a concise answer is not a style preference, it is a cost control, and capping max_tokens enforces it structurally rather than hopefully.
3. Retries multiply everything
A failed request is not free if it failed after generating. A stream that dies 30 seconds into a long answer has already produced billable output. Retry it three times and you have paid four times for one result.
Worse is retrying something that can never succeed. A 429 insufficient_quota looks like a rate limit to a naive handler, so a blanket retry-on-429 rule turns a billing failure into a loop. Classify before you retry, and cap total attempts rather than trusting backoff to terminate on its own.
4. Cache misses on a prefix you meant to cache
Prompt caching charges reads at a fraction of the base input rate, which is transformative for a long fixed system prompt. On Anthropic, cache reads cost 10% of base input. But caching keys on an exact prefix match, so anything that varies early in the prompt invalidates it.
The classic mistake is injecting a timestamp, a session ID or a user name at the top of the system prompt. Every request then has a unique prefix, every request is a cache miss, and you pay full price while believing caching is on. Put everything variable at the end, after the stable block. This is covered in full in how prompt caching cuts costs.
What to do about it
- Measure a real workload, not a single call. The API cost calculator takes your token volumes and monthly request count and ranks models by what you would actually pay.
- Cap output. Set
max_tokensdeliberately per endpoint rather than leaving one generous default everywhere. - Trim history. Summarise or drop the middle of long conversations. The context budget planner shows how fast the window fills.
- Order the prompt for caching. Stable content first, variable content last.
- Route by task. Most requests do not need your most expensive model. Compare them in the model comparison table.
Frequently asked questions
Why is my bill higher than the token count suggests?
Because the API is stateless and your client resends the entire conversation every turn. A twenty-turn exchange does not cost twenty messages, it costs the sum of a growing prefix, so a few thousand tokens of real content can mean a hundred thousand tokens sent.
Which costs more, input or output?
Output, by three to six times on every provider. Claude Opus 5 is $5 per million input against $25 output; GPT-5.6 Luna is $0.20 against $1.20. That makes verbosity the most expensive habit in a prompt, and capping max_tokens a real cost control.
Does switching to a cheaper model fix a high bill?
Usually less than expected. Model choice is normally the smallest of the four levers, behind resent history, output volume and wasted retries. Fix the workload shape first, then choose the model.
