Definition
A context window is the maximum amount of text a language model can hold at once, counted in tokens and shared by the prompt, the sources and the reply. Unlike a knowledge cutoff, which fixes what the model learned during training, the context window limits what it can read right now.
At a glance
- CategoriesTechnical
- Related fieldsAI Search
- DifficultyIntermediate
At a glance
- Everything in a conversation shares one window: instructions, history, retrieved pages and the answer.
- The unit is the token, not the word, so the same page costs different amounts in different models.
- When the window fills, something gets dropped or summarized and the system rarely says which.
- Bigger windows have not removed the problem, because retrieval adds more sources to fill them.
What counts against the window
Four things share the space: the system prompt, the conversation so far, the passages retrieved for this turn, and the reply being written. Each is measured in tokens, which are word fragments rather than whole words.
Retrieval is the greedy one. Every chunk it takes is space the next chunk cannot have, which is why systems rarely feed whole pages to a large language model. They feed selected sections. Anthropic’s documentation on context windows describes the same arithmetic from the model’s side: input and output draw on one pool.
When the pool runs short, the product truncates old turns, summarizes them, or drops lower-ranked sources. All three lose detail, and the user sees no warning.
Why your long page gets trimmed before it is read
The instinct to write one definitive guide works against you here.
A very long page does not enter the window as a long page. It enters as a few chunks, scored separately by a retrieval system. The rest was never seen. Whether your best paragraph made the cut depends on how it looked in isolation, not on whether the page is the best on the subject.
Chunks also arrive stripped of their surroundings. If your key claim relies on a definition given four sections earlier, the model gets the claim without the definition and either guesses or drops it. That gap is a common route to a hallucination about a brand.
Writing for a budget you cannot see
You cannot control the window. You can control how much of your page survives contact with it.
- Make each section self-sufficient. Define your terms where you use them, even if that means repeating yourself.
- Put the claim before the evidence. If only the start of a chunk is read closely, the start should carry the point.
- Split a long guide into linked pages where the sub-topics are genuinely separate. One sprawling page competes with itself.
- Trim the padding. Preamble and transition sentences take chunk space without adding anything quotable.
Frequently asked questions
-
Is a context window the same as a knowledge cutoff?
No. A knowledge cutoff is a date: the point after which the model learned nothing in training. A context window is a size: how much text it can read now. A model with an old cutoff can still discuss this morning’s news if that news sits inside its window.
-
Do bigger context windows mean my whole page gets read?
Rarely. Windows have grown a great deal, but retrieval fills the extra room with more sources rather than more of any single source. The chunk remains the unit selected. Assume your page is read in pieces, however large the window.
-
Does the context window affect my AI search visibility?
Indirectly, and it is worth understanding rather than optimizing for. The window is why your content gets chunked, and chunking is why self-contained sections beat long connected arguments. You cannot influence the window. You can influence how well your writing survives being cut up.
-
Should I write shorter pages because of this?
Not shorter, better divided. Length is not the problem, dependency is. A long page built from sections that each stand alone does fine. A short page whose second paragraph only makes sense after the first does badly.