What is Context window?

How it works

Every request to a model is a sequence of tokens. The context window is how long that sequence may be. Anthropic's documentation describes it as the model's "working memory": everything the model can reference when it generates a response, including the response itself. It is separate from what the model learned in training.

In a chat, each new turn re-sends the earlier turns, so the history keeps growing until it hits the limit. In agent workflows, tool definitions and tool results also count, and they can fill the window quickly. When the limit is reached, the application has to drop, summarise or compact older content, or the API rejects the request.

Sizes have grown fast, from a few thousand tokens in early models to hundreds of thousands or a million tokens in many current models. Check each vendor's model page for the exact figure, because it varies by model and sometimes by plan.

Why it matters

• Bigger is not automatically better. Research such as "Lost in the Middle" found that models use information at the start and end of a long context more reliably than information buried in the middle. Anthropic's docs call the general drop in accuracy as context grows "context rot".
• Cost and speed scale with it. You pay for every input token on every call, so filling a large window repeatedly is expensive and slower.
• It shapes system design. Deciding what goes into the window, and what stays out, is the core of context engineering and the reason RAG exists.

Example

You load a 400-page manual (about 200,000 tokens) into a model with a 1M-token window and ask about one setting. It fits, but a better approach is often to retrieve the three relevant pages and send only those: the answer tends to be more accurate, faster and far cheaper.