New major model releases regularly promise bigger context windows. Sounds great until you realize it's mostly marketing. A bigger context window doesn't mean better results. It often means more data, more noise, and more AI slop.
According to multiple studies, models effectively use only about 30–50% of their available context. For example, a model with a 200K-token context window may already show noticeable quality loss at around 50K tokens.
Why this happens:
🔸 Context rot. Output quality gradually degrades as the context grows.
🔸 Reasoning shift. The model spends less effort on reasoning. The answers sound more confident, but their quality often gets worse.
🔸 The lost-in-the-middle effect. Information in the middle of the context can be overlooked during later reasoning.
🔸 Attention dilution. The model's attention is spread across different instructions, making it harder to focus on what actually matters.
The practical takeaway is simple: keep your context clean:
🔸 Start a new conversation for each new task (
/new in Claude).🔸 During long-running tasks, use /compact regularly to collapse intermediate reasoning and keep only the important things.
🔸 Store large data in long-term memory or relevant documentation, and bring it into the context only when it's actually needed.
Useful references:
- https://www.morphllm.com/context-rot
- https://www.zenml.io/llmops-database/context-rot-evaluating-llm-performance-degradation-with-increasing-input-tokens
- https://arxiv.org/html/2601.11564v1
#engineering #ai #tips