Blog

AI Context Window Explained: What It Is and Why It Matters

Updated June 2026 · 9 min read

If you have ever had an AI chatbot suddenly "forget" what you told it ten messages ago, or received an error saying your input was too long, you have hit the context window limit. The context window is one of the most important — and most misunderstood — concepts in AI. This guide explains exactly what it is, how it works, and what you can do to work within its constraints effectively.

What is a context window?

The context window is the maximum amount of text an AI language model can read and process in a single interaction. Think of it as the model's short-term memory: everything the model can "see" right now — your instructions, the conversation history, any documents you pasted in, and its own previous responses — must fit inside this window.

When you open a fresh chat with ChatGPT or Claude, the context window is empty. As the conversation grows, it fills up. Once it reaches the limit, the model can no longer see the oldest content — it has effectively been "pushed out" of memory. This is why a long enough conversation can cause an AI to contradict what it said earlier or forget facts you shared at the start.

The context window is measured in tokens, not words. A token is roughly 3–4 characters of English text, meaning 1,000 words is approximately 1,300–1,500 tokens. To understand tokens in depth, see our guide: What Are AI Tokens?

How context windows work

Understanding the mechanics of a context window requires knowing what goes into it and how resets happen.

Input + output = total tokens

The context window is shared between everything sent to and received from the model. Specifically, it contains:

System prompt — the instructions that configure the AI's behavior (set by the developer or app)
Conversation history — every prior turn of the chat (both user and assistant messages)
Your current message — what you typed right now
The model's response — the output the model generates

All of these combined must stay below the model's context limit. If you have a 128,000-token context window and your conversation history already uses 120,000 tokens, the model only has 8,000 tokens left for your new message and its response.

The context window resets with each API call

AI models are stateless by default. When you call the API directly, each request is independent — the model has no memory of previous requests unless you explicitly include the prior conversation in your payload. Chat applications like ChatGPT handle this automatically by appending each new turn to the history they send with every API call. But at the API level, the "context" is just text you pass in, nothing more.

This is a crucial point: the context window is not stored in the cloud between sessions. It exists only for the duration of one API call. Everything outside the current window is gone from the model's perspective. See our tokens per word guide for help estimating how quickly your context fills up.

Context window sizes across major models

Context window sizes have grown dramatically since GPT-3's 4,096-token limit in 2020. Here are the current limits for the most widely used models (as of June 2026):

Model	Context Window	≈ Words
GPT-4o	128,000 tokens	~96,000
GPT-4o mini	128,000 tokens	~96,000
Claude Sonnet 4	200,000 tokens	~150,000
Gemini 2.0 Flash	1,000,000 tokens	~750,000
Gemini 2.0 Pro	1,000,000 tokens	~750,000
GPT-3.5 Turbo	16,000 tokens	~12,000

Gemini's 1-million-token context window is the largest commercially available at the time of writing — enough to fit approximately 750 full short stories. Claude's 200K window is particularly popular for document analysis and long-form coding tasks. Use our token calculator to estimate how much of any model's context window your content will consume.

Why context window size matters

The practical impact of context window size shows up most clearly in three types of workflows.

1. Processing long documents

Suppose you want an AI to summarize a 50-page legal contract or review an entire codebase. A 16K-token model (about 12,000 words) cannot fit a typical contract — you would need to chunk it manually and process it in pieces, losing cross-document coherence. A 200K model can hold the entire document in one pass, allowing it to understand references, contradictions, and structure that span the whole document.

2. Multi-turn chat and customer support

In conversational applications, every message adds to the history. A customer support bot that has handled fifty back-and-forth messages with a user is consuming tens of thousands of tokens just in the conversation log, before the system prompt or reference materials even come into play. A larger context window means the AI can sustain longer, more coherent conversations without losing track of earlier details.

3. Code review and software development

A typical software repository might contain hundreds of files. When using AI for code review or refactoring, being able to send multiple related files in a single prompt — rather than one file at a time — dramatically improves output quality. The model can spot inconsistencies across files, check that interfaces match their implementations, and maintain a coherent understanding of the system architecture. See our comparison of GPT-4o vs Claude for how these models differ in code-related tasks.

What happens when you exceed the context window

The behavior when you hit the context limit depends on whether you are using a chat interface or the API directly.

API: explicit error

When calling the API directly, exceeding the context window returns an error. OpenAI returns a context_length_exceeded error. Anthropic returns an invalid_request_error with a message about the prompt length. Your application must handle this — typically by shortening the input or implementing a summarization strategy.

Chat UIs: silent truncation

Consumer interfaces like ChatGPT and Claude.ai handle this more gracefully from the user's perspective — but it comes with a hidden cost. When the conversation history grows too long, the app silently drops the oldest messages from the context before sending the API request. From your perspective, the conversation appears continuous; from the model's perspective, those early messages never existed.

This is the source of the frustrating experience where an AI "forgets" something you told it thirty messages ago. It did not actually forget — it was never given that information in the current API call. The message was truncated to keep the total token count below the limit.

How to manage your context window effectively

Working intelligently within context window limits is a key skill for anyone building with AI. Here are the most effective strategies.

Summarize conversation history

Instead of keeping every message verbatim, periodically ask the model to summarize the conversation so far. Replace the full history with this compact summary. You retain the essential information while freeing up thousands of tokens for new content. Many production chatbot frameworks do this automatically at configurable thresholds.

Chunk large documents

For documents that exceed even large context windows, use a chunking strategy: split the document into overlapping segments (overlap ensures context is not lost at boundaries), process each chunk independently, and then synthesize the results. The overlap size is typically 10–20% of the chunk size.

Use retrieval-augmented generation (RAG)

Rather than putting entire document libraries into the context window, use a vector database to retrieve only the most relevant passages for each query. This is called Retrieval-Augmented Generation (RAG). The model only sees the retrieved snippets — a few hundred to a few thousand tokens — rather than thousands of pages. This scales far beyond what any context window can accommodate.

Trim system prompts and instructions

Long system prompts eat into your available context. Audit your system prompts regularly — remove redundant instructions, consolidate repetitive rules, and keep only what is actually necessary for the task. Every token saved in the system prompt is a token available for user content.

Use the right model for the job

Not every task needs a million-token context window. For simple, short interactions, a model with a 16K or 32K window is cheaper and often faster. Reserve large-context models for tasks that genuinely need them — multi-document analysis, long code reviews, extended research sessions.

Context window vs memory: an important distinction

The terms "context window" and "AI memory" are often used interchangeably in popular writing, but they refer to different things — and the distinction matters.

The context window is temporary. It is the active working space for a single session or API call. When that call ends, the context is gone. Nothing is retained between sessions unless you explicitly save and reload it.

AI memory — in the sense that some products like Claude.ai and ChatGPT now implement — is a separate system layered on top of the base model. It works by storing facts and summaries in a database outside the model, then automatically injecting relevant memories into the context window at the start of new sessions. The underlying model still has no inherent long-term memory; the persistence is implemented by the application layer, not the model itself.

This means: if you share your name, preferences, or project context with an AI and then start a completely new API-level session without providing that information again, the model will not know it. The context window is blank. Any apparent "memory" in consumer products is an engineered feature, not a native capability of the language model.

Estimate your context usage

Use our free Token Calculator to see exactly how many tokens your text uses and whether it fits within different models' context windows.

Open Token Calculator →

Frequently asked questions about AI context windows

What is a context window in AI?

A context window in AI is the maximum amount of text — measured in tokens — that a language model can process in a single interaction. It includes the system prompt, conversation history, your input, and the model's output. Everything that the model can "see" and reason about must fit inside this limit. Think of it as the model's short-term working memory for a single session.

What is the largest AI context window?

As of June 2026, Google's Gemini 2.0 Flash and Gemini 2.0 Pro both support 1,000,000-token context windows — equivalent to roughly 750,000 words or about ten average-length novels. Claude Sonnet 4 offers 200,000 tokens, and GPT-4o offers 128,000 tokens. Context window sizes continue to grow with each new model generation.

Does ChatGPT have a context window?

Yes. ChatGPT uses GPT-4o, which has a 128,000-token context window — roughly 96,000 words. In the chat interface, OpenAI automatically manages this by silently dropping older messages when the conversation grows too long. At the API level, you control exactly what goes into the context and must handle the limit yourself. ChatGPT's "memory" feature is a separate system that stores facts between sessions and is distinct from the context window.

Why does my AI forget things?

When an AI appears to forget something you told it earlier, one of two things has happened: either the conversation grew long enough that older messages were silently dropped from the context window to make room for newer content, or you started a new session and the prior conversation was never included in the current context. AI models have no persistent memory by default — they only know what is in the current context window. If you need the AI to remember something across a long conversation, repeat the key facts periodically or use a summarization strategy.

How do I increase my context window?

You cannot increase a model's built-in context window limit — it is a fixed architectural property of the model. However, you can make better use of your available context by: (1) compressing conversation history into summaries, (2) removing boilerplate from your system prompt, (3) using retrieval-augmented generation to avoid pasting large documents into the prompt, and (4) choosing a model with a larger native context window for tasks that genuinely require it — such as Claude Sonnet 4 at 200K or Gemini 2.0 at 1M tokens.