Google presented Dream-RSI

The central idea is almost obvious in retrospect: that history is already a simulator.

Want to know if a different exploration strategy would have worked better? You don't need to re-run anything. You walk the same recorded tree in a different order. Every outcome is already there. Cost: zero executions.

The architecture has 3 layers:
The underlying coding agent stays completely untouched. On top of it sits a lightweight orchestration harness an executable policy that controls branching decisions, parallel exploration, and stopping rules.

Above that, a policy development agent rewrites the harness code between rounds, testing each new version against the replay simulator before any of it ever runs live.

This separation is the key design choice. The harness makes exploration programmable without touching the model underneath. And because the harness is just code, it can be evaluated, revised, and selected offline thousands of candidate versions screened against accumulated history before a single one ships.

The loop: 1. The agent explores online and builds a discovery tree 2. That tree becomes a replay simulator 3. Thousands of alternative harness policies are tested inside it no real runs, just traversals 4. The best harness is redeployed, records a new tree, expands the pool.