Large language models concept

Context windows explained

Reference

A context window is the amount of information a language model can use while producing a response. It is measured in tokens, small text units that can be whole words or parts of words. The window can include your message and other material supplied with it, plus room for the model’s reply. It is a limit on the information available for a response, not a promise that every detail inside it will be noticed or recalled.

Reviewed 2026-10-09

A working space for one response

Think of the context window as a desk holding the material available while a model works on a reply. That material can include your question, earlier messages, instructions, and other supplied content. The model’s reply also uses part of the available space. The exact limit depends on the model, and token counts do not match word counts one for one: a token can be a whole word, part of a word, or even a single character.

This is why “a larger window” means the model can be given more material at once, not that it has a permanent record of every conversation. A context window describes what is available for a response. A separate memory feature, when a product offers one, may carry selected details across chats; its behavior depends on that feature.

How the information gets used

First, a chat service assembles the material for a turn. In a typical API request, that can include prior messages, the latest question, instructions, tool descriptions, and tool results. Second, the text and other supported inputs are converted into tokens, which are the units counted against the window. Third, the model processes the available sequence and generates a reply; in a Transformer, attention mechanisms relate positions in a sequence as it builds representations.

That reply takes up space too. In a multi-turn conversation, the service may include earlier messages again when preparing a later turn. So each turn uses the available capacity for both the supplied input and the generated output, within the model’s particular limit.

Room is not the same as recall

A common mistake is to treat the window as a guarantee that the model will use every detail in it. A large window gives more room for material, but performance can vary with the context, and a model can miss or misread information that was supplied. Keeping relevant details clear and putting the question near them can make the task easier to inspect, but it does not guarantee a correct answer.

Another trap is assuming a chat remembers everything forever. A service may leave out, summarize, or otherwise manage old conversation content as a conversation grows. That is a product behavior, not the definition of a context window. If a key detail matters, check whether it is available in the current exchange and restate it when needed.

Check what you’ve understood

Try this with a chat you use: identify the latest question, any instructions, earlier messages or attached material, and the reply the model is about to produce. Ask yourself which parts are supplied for this response and which details, if any, come from a separate memory feature. Then consider what might happen if the conversation keeps growing: would the service include all earlier content, or might it manage that content to stay within a limit?

You have the distinction if you can say: the window is the model’s available working context for a response; memory is a separate way a product may retain or retrieve information beyond that immediate input. Neither label by itself tells you that the model will recall a particular detail correctly.

Questions people ask

What counts toward a context window?

It depends on the service, but an API request can count instructions, conversation messages, tool details and results, and supported non-text inputs. The generated reply can count too. Check the service’s documentation for its exact rules.

Does the context window include the model’s answer?

Often, yes. For example, Google describes the limit as combining input and output tokens, and Anthropic says its window includes the response being generated. Exact accounting depends on the service.

Is a context window the same as memory?

No. The window is the information available for a response. A product’s memory feature may preserve or retrieve selected details across chats, but it is separate and may not retain every detail.

If a model has a large context window, will it remember everything I put in?

No. A larger window allows more input, but it does not guarantee that the model will find or use every detail correctly. Performance can vary with the context.

Learn more

Understand and count tokens
Explains tokens and how input and output are counted toward a model’s context limit.
Context windows
Shows how conversation history, generated output, and other request components can use context capacity.

What changed

Sources

Part of Large language models: From text to tokens and back Where fluent output can fail