Figures on this page were verified 31 August 2026 against the providers' own documentation. Pricing, context windows and rate limits change without notice, so confirm any number against the provider before you rely on it. Tell us if something here is out of date.
When a model “forgets” something you told it, the information almost always left the context window rather than being misunderstood. Conversations grow, older turns get dropped, and the model answers from what remains. The fix is structural: put durable facts somewhere that is resent every time, and keep the volatile part small.
Three tiers of context
| Tier | Contains | Lifetime |
|---|---|---|
| Durable | Stack, conventions, constraints, commands | Months; changes with the project |
| Task | What you are doing right now, relevant files | Hours; changes per task |
| Turn | The current question and recent exchange | Minutes |
Almost every “it forgot” complaint is durable information that was only ever stated in a turn. Said once in message three, it is gone by message forty. Put it in the durable tier and it is present in every request, which is why a committed rules file is worth more than repeating yourself.
Order the prompt so caching works
The tiers should appear in the prompt from most stable to most volatile. This is partly for the model’s benefit and mostly for your bill’s: prompt caching matches on an exact prefix, so anything variable near the top makes every request a cache miss.
# Stable prefix first, so it can be cached across every request
messages = [
{"role": "system", "content": [
{"type": "text", "text": PROJECT_RULES, # durable
"cache_control": {"type": "ephemeral"}},
{"type": "text", "text": current_task_brief}, # task
]},
*recent_turns, # volatile, last
]Getting this order wrong is the most common reason teams believe caching “does not work for us”. It works; a timestamp or session ID at the top was defeating it.
Curate files, do not dump directories
The instinct when a model gets confused is to add more files. Past a point this makes answers worse: the relevant code is diluted, cost rises, and you approach context_length_exceeded.
Include the file being changed, its direct callers, and the type or interface definitions it depends on. Exclude generated code, lock files, vendored dependencies and test fixtures unless the task is about them. Most tools honour an ignore file, and configuring it is usually the single highest-leverage change to answer quality.
Summarise instead of truncating
When a conversation approaches the window, the default behaviour is usually to drop the oldest turns. That silently discards the decisions made early, which are often the ones that still matter.
Summarise the middle instead. Keep the system prompt and the most recent turns verbatim, and replace what is between them with a short factual digest: decisions taken, files changed, constraints discovered. A 300-token summary preserves more usable signal than 3,000 tokens of raw transcript.
To see how many turns you actually have before this becomes necessary, put your system prompt, tool definitions and max_tokens into the context budget planner.
Start a new session more often
A long session is not a virtue. Once a task is finished, its transcript is pure cost on every subsequent request, and it actively degrades answers by surrounding the current question with irrelevant history. If the durable tier is written down properly, starting fresh costs you nothing, which is the real reward for maintaining a rules file.
Frequently asked questions
Why does the model forget things I told it?
Because the information left the context window rather than being misunderstood. Anything said once in an early turn is gone by turn forty. Durable facts belong somewhere resent on every request, such as a committed rules file.
In what order should context appear in the prompt?
Most stable first, most volatile last. Prompt caching matches on an exact prefix, so a timestamp or session ID near the top makes every request a cache miss. This is the usual reason teams conclude that caching does not work for them.
Is it better to truncate or summarise a long conversation?
Summarise the middle. Dropping the oldest turns silently discards the decisions that often still matter, whereas a 300-token digest of decisions and constraints preserves more usable signal than 3,000 tokens of raw transcript.
