If an AI agent gives a wrong answer, the instinct is almost always the same: give it more context. More background, more documentation, more of the conversation history, more tools it could use. More information can only help, right?
It turns out that instinct is wrong more often than most people realize. A growing body of research from 2025 and 2026 shows that past a certain point, adding more context doesn't just fail to help. It actively makes the model worse, on the exact task you gave it more information to solve.
The phenomenon now has a name: context rot
In July 2025, the AI infrastructure company Chroma published a study testing 18 frontier models, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3, on how their performance changed as input length grew. The researchers call the pattern context rot.
The finding, in short: model reliability decreases significantly with longer inputs, even on simple tasks like retrieval and text replication. Not exotic reasoning tasks. Simple ones. Tasks a model should, in principle, handle identically at 1,000 tokens and 100,000 tokens.
It doesn't. And the reason why breaks down into a few distinct, independently confirmed effects.
Length alone hurts, even without distractions
The first surprising result is that context length itself degrades performance, separate from whatever is actually in that context. A 2025 study published at the ACL Findings conference tested this directly by inserting 25,000 blank characters, white space with no real content at all, into a model's input.
Even with almost no actual distracting information, simply making the input longer caused measurably worse reasoning performance.
The model could still correctly extract every fact and condition it needed. It just reasoned worse once it had to do so inside a longer input, regardless of what filled that extra length.
Irrelevant content actively misleads the model
Length alone is one problem. What you put in that length is a second, larger one. Chroma's research found that semantically similar but irrelevant content actively misleads a model, not just wastes its attention. A single irrelevant passage was enough to measurably reduce accuracy, and adding more distractors compounded the damage further.
A separate November 2025 analysis of LLM behavior at scale found the effect is even sharper than "somewhat worse":
A single irrelevant passage can reduce accuracy by up to 30%, and the amount of damage correlates directly with how semantically similar that irrelevant passage is to the real answer.
This last part matters. A random, obviously unrelated paragraph is easy for a model to ignore. A paragraph that sounds relevant, uses similar vocabulary, and sits near the real answer in meaning is exactly the kind of "helpful extra context" a well-intentioned system would add, and it's exactly the kind that does the most damage.
The most counterintuitive finding: coherent text is worse than scrambled text
Here's the part of the research that genuinely surprises most people who hear it for the first time.
Chroma ran an experiment comparing two versions of the same long document used in a standard retrieval test. One version kept the natural, logical flow of the original writing. The other took the exact same sentences and shuffled them into random order, destroying any coherent structure while keeping the same overall topic.
The expectation, reasonably, was that the coherent version would be easier. A logically flowing essay should make an inserted, out-of-place fact stand out more, like a typo in a well-edited paragraph.
The actual result was the opposite.
Across all 18 models tested, performance was consistently better on the shuffled, incoherent version of the text than on the naturally flowing one.
Structural coherence, the very thing that makes text easier for a human to read, made it harder for these models to retrieve a specific fact from within it. Researchers suspect this points to something real about how attention mechanisms process input: a logically flowing passage may generate more plausible-looking distractors along the way, things that read as "this could be relevant" precisely because the writing is coherent enough to seem connected.
Bigger context windows don't fix this. They can make the exposure worse
It's tempting to assume this is a today problem that newer, larger context windows will simply outgrow. The research says otherwise.
A 2025 analysis introduced the concept of a model's Maximum Effective Context Window (MECW), the point where a model's real accuracy starts falling off, separate from its advertised maximum. The findings were stark:
- Some models showed clear accuracy degradation by as little as 1,000 tokens, far below any advertised limit in the hundreds of thousands.
- Across the models tested, actual effective context fell short of the advertised maximum by more than 99% in some cases.
This isn't limited to text retrieval either. Berkeley's Function-Calling Leaderboard found the same pattern in tool use: models given too many tools to choose from perform measurably worse, with degradation beginning once the tool count exceeds roughly 30 options, as descriptions start overlapping and the model struggles to discriminate between them.
Even labs building these models have said as much about their own systems. Anthropic's own research indicates that contexts larger than 100,000 tokens can degrade reasoning quality on Claude models, and Meta's Llama 3.1 405B shows measurable performance decline starting around 32,000 tokens. More capacity does not automatically produce better reasoning, even from the people who built the model.
Why this happens: attention wasn't built to filter, it was built to cover
The mechanistic explanation researchers point to comes down to how these models are trained in the first place. The core capability that lets a model reason over context, multi-head self-attention, is optimized during training to maximize the likelihood of the next correct token given everything it's been shown.
That objective rewards coverage, weighing everything in context somewhat, not discrimination, confidently ignoring the parts that don't matter. The practical result: attention weights don't reliably encode how confident or relevant a given piece of context actually is. Everything gets some attention, including the content that should have been filtered out before it ever reached the model.
What this actually means in practice
None of this is an argument for giving a model less information across the board. It's an argument for being deliberate about what actually reaches the model, rather than treating "more context" as a free, universally safe default:
- Relevance beats volume. A shorter input containing only what's actually needed will often outperform a longer one padded with tangentially related material, even when that material is technically accurate.
- Semantically similar but irrelevant content is the most dangerous kind. It's the hardest for a model to filter out on its own, and the easiest for a human or a retrieval system to accidentally include because it "seems related."
- Retrieval and filtering are doing real work, not just saving tokens. A system that decides what to hand a model, rather than handing it everything and hoping attention sorts it out, is solving a problem the model's own architecture cannot reliably solve for itself.
The instinct to add more context comes from a reasonable place. More information feels like it should never hurt. The research from the past two years says clearly that it does, consistently, across model families, context lengths, and task types. Curating what a model sees isn't a workaround for a smaller context window. It's a real, separate discipline that even the largest context windows still need.
