Published a paper on Recursive Language Models (RLM), and this essentially solves the "context decay" problem that plagues even the most powerful models like GPT-5.
🟢 How DeepMind solved the problem?
Instead of trying to "hold in memory" 10 million tokens in a single attention window, RLM treats the prompt as an external variable in a Python REPL. The model doesn't read the entire text; it "navigates" through it.
HOW IT WORKS?
The model writes code to perform grep, slice fragments, and recursively call sub-instances of itself on relevant pieces of data.
Ideal memory: When the context is externalized, the model maintains 100% accuracy regardless of document length.
EMERGENT BEHAVIOR: Without special training, the models started using regex to filter data and build recursive "check and correct" loops.
CHEAPER and FASTER: Since it only "reads" the small fragments that are actually needed, the median cost is often lower than that of regular calls with large context.
RESULTS (on Multi-Doc Research):
→ GPT-5 Base: 0% (crashed/failed)
→ GPT-5 + RLM: 91%
→ Reasoning on dense data:
→ Base: 0.04%
→ RLM: 58%
This is a complete shift from "making windows bigger" to "making navigation smarter".
Paper
••••••••••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore