The Context Window is the maximum amount of input data (measured in tokens) an AI model can 'remember' and actively process at one time when generating a respon
The context window is the maximum tokens a model processes per request: prompt plus generated output combined. Everything the model 'knows' at inference must fit inside it: instructions, retrieved documents, conversation history, tool results. Attention costs grow with length, so bigger windows raise latency and price even as 2026 models stretch into hundreds of thousands of tokens (with million-token frontiers).
Context is the working memory budget of every AI application. Its limits drive core architecture decisions: what to retrieve versus summarize, how agents persist memory across steps, how costs scale with conversation length. 'What's in context?' is the first debugging question for most quality issues.
No: long contexts cost more, run slower, and models can attend unevenly across them ('lost in the middle'). Curated, relevant context typically beats indiscriminate stuffing; retrieval plus a moderate window remains the workhorse pattern.
Requests fail or oldest content is truncated: silently degrading behavior if unmanaged. Production systems count tokens, summarize or evict history deliberately, and persist long-term memory outside the window.
No. Even huge windows can't hold enterprise corpora, and cost/latency scale with tokens processed. Retrieval selects what deserves the window; the two are complements, not competitors.