What Is Context Rot?

Context rot is the tendency for a model's response quality to decline as more content gets loaded into its context window, even when the total is nowhere near the window's stated limit. A model might technically accept a hundred thousand tokens of input, but that does not mean it uses every part of that input equally well. As the amount of text grows, the model's ability to notice, weigh, and correctly use the specific detail that actually matters tends to get worse, even though nothing has been cut off or truncated.

The reason this happens comes down to how attention works inside these models. A model has to decide, for every piece of output it generates, how much weight to give to every piece of the input, and that job gets objectively harder as the input grows and fills with more competing detail. A short, focused prompt makes it easy for a model to give the right things the right weight. A long prompt packed with mostly irrelevant material, or with the truly important sentence buried somewhere in the middle rather than near the start or end, makes that same job meaningfully harder, and errors creep in.

What separates context rot from simply running out of context is that nothing here is actually failing outright. Hitting a hard token limit produces an obvious error or forces truncation, an easy problem to notice because something visibly breaks. Context rot is quieter: the request succeeds, the model returns a fluent, confident-sounding answer, and that answer is subtly wrong or missing a detail that was technically present in the input the whole time. The naive assumption, that a model will use everything you give it equally well as long as it fits under the limit, is exactly the assumption context rot breaks.

By 2026, context windows have grown enormously, with models routinely accepting hundreds of thousands of tokens, and it is tempting to treat that growth as having solved the problem of giving a model enough information. Bigger windows did not eliminate context rot, though. Teams that dump entire document sets, full chat histories, or bloated system prompts into every request still see performance wobble on the details that get lost in that mass of text, which is why disciplined context management remains a real engineering skill rather than something a bigger window makes unnecessary.

This page covers what is actually happening when context rot shows up, how it compares to the more obvious failure of running out of context window entirely, what separates it from catastrophic forgetting, a different kind of degradation that happens during training rather than at the moment of use, and how to keep a system from being quietly undermined by it. The idea worth keeping is that a bigger context window buys you room, not attention, and those are not the same thing. Keep that distinction in view throughout, since it explains why fixing one of these problems does nothing at all to fix the other.

• Context rot is a decline in response quality as more content fills a model's context window, even when the total stays well under the stated limit.
• It happens because giving appropriate attention to specific details gets objectively harder as the input grows longer and more crowded with competing material.
• Unlike running out of context, which fails loudly and obviously, context rot fails quietly, producing a confident answer that is subtly wrong or missing something.
• Larger context windows by 2026 have not eliminated the problem, since accepting more tokens is not the same as attending to them equally well.
• A bigger context window buys more room to fit information, not better attention to that information, and treating the two as the same thing invites quiet mistakes.

Context rot tends to show up first as a drop in accuracy on details that sit somewhere in the middle of a long input, rather than near the beginning or the end. A model asked to find one specific fact buried deep inside a long document will often do noticeably worse than the same model asked to find that same fact when it sits near the start or the end of the same document, even though the fact itself did not move and the total length did not change.

It also shows up as increasing difficulty juggling multiple pieces of information at once. A single, clearly stated fact somewhere in a long context is one thing to retrieve. Correctly combining several separate details scattered across that same long context, especially when some of them seem only loosely related to the question being asked, tends to be where quality drops off faster and more noticeably. This is exactly the kind of task that long-context marketing tends to imply a model handles effortlessly, when in practice it is one of the harder things to get right.

Irrelevant or redundant content in the context tends to make things worse even when the actually relevant material is technically all still there. A long context padded with repetitive boilerplate, outdated information that no longer applies, or content unrelated to the current question adds noise for the model to sift through, and that noise competes for the same limited attention that the genuinely useful content needs in order to be used correctly. A context window that is technically well within limits can still be functionally cluttered in a way that quietly undermines the very task it was assembled to support.

The failure itself rarely looks like an obvious error. A model experiencing context rot typically still produces a fluent, well-formed, confident-sounding response, it just quietly gets a detail wrong, misses something it was given, or blends together two separate pieces of information that should have stayed distinct. That fluency is exactly what makes the problem easy to miss without deliberately testing for it, since a wrong answer delivered smoothly reads the same as a right one to a reader who is not checking closely.

Running out of context window is the more obvious failure. It happens when the input, or the input plus the desired output, exceeds the model's stated limit, and the result is usually an outright error, an automatic truncation of some of the input, or a request getting rejected before it even runs. Something visibly breaks, and it is generally clear where the problem happened and roughly why. Fixing it is usually mechanical too, since you can see exactly what got cut and adjust accordingly without much guesswork.

Context rot happens well inside that limit, and it produces no error at all. The request completes normally, every part of the input was technically available to the model, and the response comes back looking perfectly ordinary. The only sign anything went wrong is that the answer is quietly off in a way you have to check for specifically, usually by verifying it against the source material rather than by noticing anything unusual in how the system behaved. That absence of any visible signal is precisely what makes it so easy for a team to ship something quietly broken and not find out for a long while.

The practical difference is how you find each one. Running out of context window announces itself, since you get an error message or a visibly truncated input, and the fix is straightforward: shrink the input or increase the limit. Context rot has to be found through actual evaluation, testing whether the model correctly used specific details placed at different points inside a long context, because nothing in the system's behavior will flag the problem on its own. Building that kind of evaluation into a regular process, rather than treating it as a one-time check, is really the only durable defense against the problem.

Both problems point toward the same underlying lesson even though they fail differently: more context is not free, and it is not purely a matter of whether something fits. One failure mode punishes you for exceeding a hard boundary. The other quietly punishes you for staying well inside it but asking the model to make good use of too much material at once. Treating a large context window as an unqualified upgrade, without accounting for either failure mode, is the mistake that quietly connects both of them.

Context rot and catastrophic forgetting both involve a model losing track of something, which is why the two get mixed up, but they happen at completely different stages of a model's life. Catastrophic forgetting is a training-time phenomenon, where a model being trained or fine-tuned on new data loses accuracy on things it previously learned well, because the new training pushes its weights away from what it knew before. Keeping the vocabulary straight matters because the fix for one has essentially nothing to offer the other, no matter how similar the symptoms sound at first.

Context rot has nothing to do with training at all. It happens purely at the moment a model is being used, within a single request, as a function of how much content is sitting in that one context window at that one moment. The model's underlying weights, and everything it learned during training, are completely unaffected. Nothing about the model has changed. What changed is only how much material it is being asked to work with right now. The same model, unchanged in every respect, can look sharp on a short prompt and noticeably shakier on a long one purely because of what it is being asked to juggle in that moment.

The fixes for the two problems look nothing alike because the problems live in different places. Catastrophic forgetting is addressed through training techniques, things like mixing old and new data during fine-tuning or using methods designed specifically to protect previously learned knowledge. Context rot is addressed at the point of use, through how a prompt and its supporting material are structured and curated for a specific request, with no retraining involved at all. Spending engineering time on the wrong category of fix is a common, entirely avoidable mistake once the distinction is actually understood.

Confusing the two leads to solving the wrong problem. A team that notices a model behaving worse on long contexts and assumes it must be some kind of forgetting from training will look in entirely the wrong place, since retraining changes nothing about how attention spreads across a single long input at inference time. The actual fix lives in how context is assembled and trimmed for that request, not in the model's training history. A far more productive question to ask first is simply how the specific request's context was built, what it contains, and whether that content was curated at all.

Context rot matters most for anything that pulls in large amounts of supporting material and expects precise use of specific details within it, such as legal or contract review, technical documentation lookup, or long research conversations where earlier details need to stay accurate many turns later. In these cases, a subtly wrong answer can be genuinely costly, and the fluency of the wrong answer makes it easy to miss. Any workflow where a wrong but confident answer could plausibly slip past a busy reviewer deserves this kind of scrutiny before it ships.

It also matters for systems that automatically stuff large amounts of retrieved content into every request without much curation, on the theory that more material can only help. That theory is exactly backward once context rot is accounted for, since piling on marginal or redundant material can crowd out the model's attention to the parts that actually matter, even while technically giving the model more to work with. A retrieval step that returns fewer, better-chosen results will often outperform one that simply returns everything even remotely related to the question.

It matters less for short, focused interactions where the context is naturally small and directly relevant to the question being asked, since there is little room for attention to get spread thin when there is not much competing material in the first place. A short, well-targeted prompt is close to immune to this problem by construction. These are exactly the kinds of interactions where worrying about context rot at all would be solving a problem that was never actually present.

It also matters less for use cases where an occasional missed detail carries low stakes and gets caught quickly by a human reviewing the output anyway, such as casual brainstorming or early drafts that will get checked and revised regardless. The risk of context rot scales with how much you are relying on the model to get every detail right unsupervised, not with the mere presence of a long context. The right amount of caution here should scale with the stakes of being wrong, not with some fixed rule about how long a context is allowed to get.

Curate what actually goes into a context window instead of defaulting to including everything that might conceivably be relevant. Trim redundant, outdated, or tangential material before it goes in, since every extra token is competing for the same limited attention as the content that genuinely matters, and cutting the noise is often more valuable than adding more signal. Treat this curation step as an ongoing habit rather than a one-time cleanup, since new noise tends to accumulate again the moment nobody is paying attention.

Place the most important information near the beginning or the end of the context where possible, rather than assuming position does not matter. Content buried in the middle of a long context is where models most consistently struggle, so structuring a prompt to put critical facts or instructions at the edges, rather than sandwiched in the middle of supporting material, can measurably improve how reliably they get used. This is a low-cost change to make in almost any prompt template, and it tends to pay off disproportionately relative to how little effort it takes.

Use retrieval to pull in only what is relevant to a specific question rather than loading an entire document set into every request by default. A well-targeted retrieval step that returns a focused, relevant slice of material tends to outperform simply handing over everything and hoping the model finds what it needs inside a much larger, noisier context. The instinct to include more just in case is understandable, but it usually works against you once a model has to sift through the extra material.

Test long-context performance directly rather than assuming it is fine because the model accepts a large window without complaint. Build evaluations that place specific facts at different positions within a long context and check whether the model retrieves and uses them correctly, since this is the only reliable way to know whether context rot is actually affecting your specific use case and how badly. Repeat this kind of test periodically, since changes to a model or a prompt template can quietly shift how well it handles the same long inputs over time.

Summarize or compress older material in long-running conversations instead of letting raw history accumulate indefinitely. A running conversation that keeps every previous message verbatim keeps growing the context and, with it, the risk that something important from early on gets lost in the crowd. Periodically condensing that history into a shorter summary keeps the truly relevant information present without letting the surrounding noise pile up unchecked. Done well, this keeps a long-running conversation feeling coherent to the model without forcing every earlier detail to compete for the same limited attention indefinitely.

• Trim redundant or tangential material from a context window rather than including everything that might conceivably be relevant.
• Place the most critical information near the beginning or end of a long context rather than burying it in the middle.
• Use targeted retrieval to pull in only the relevant slice of material instead of loading an entire document set by default.
• Test long-context performance directly with facts placed at different positions, rather than assuming a large window means reliable use of everything in it.
• Summarize or compress older conversation history periodically instead of letting raw context accumulate indefinitely.

• Context rot is not the same as running out of context window; it happens well inside the limit, with no error and no truncation.
• A bigger context window does not fix context rot on its own; accepting more tokens is not the same as attending to all of them equally well.
• Context rot is not catastrophic forgetting; it happens at the moment a model is used, not during training, and it does not change the model's weights.
• More supporting material in a prompt is not automatically better; irrelevant or redundant content can crowd out attention to what actually matters.
• Context rot does not produce an obviously broken response; the output usually looks fluent and confident even when it is quietly wrong.