Google and DeepMind’s Dream-RSI lets agents improve by dreaming on search trees

Most recursive self-improvement stories assume the model must rewrite its own weights. Dream-RSI argues the expensive part is not the coder. It is the exploration policy that decides where to search next. In a paper submitted September 14, 2026, researchers from Google, Google DeepMind, the University of Maryland, and the University of Virginia introduce an orchestration layer that leaves the underlying coding agent unchanged. Instead, it turns finished discovery trees into a replay simulator and lets new exploration policies “dream” through that history before spending another real rollout.

History as an exact simulator

The core insight is almost bookkeeping. A long discovery run already records a tree of attempts and their execution outcomes. An exploration policy only chooses which branch to continue, what to parallelize, and when to stop. So an alternative policy does not need to re-run the world. It can walk the same recorded tree in a different order and read outcomes that are already on disk. Dreaming means screening thousands of candidate policies against that realized search space at zero extra executions, then deploying only the winner online.

The project site puts it plainly: history is the world the agent dreams in. The simulator is not a learned approximation. It is exact over the search space that was actually visited. Its limit is equally sharp. A policy can only be dreamt where history went, which is why the loop must keep adding new online trees.

The RSI loop without weight surgery

Each outer iteration deploys the current exploration policy to guide a fixed discovery agent, appends the resulting tree to history, then runs an offline policy-development agent that rewrites the exploration policy and scores revisions by replay. Because the currently deployed policy stays in the candidate set, the winner is never scored worse than what it replaces. The improved policy goes back online, expands the simulator pool, and the recursion continues.

Across algorithm engineering, mathematical optimization, and GPU kernel engineering, the authors report competitive or better discovery quality at substantially lower discovery cost. On a Lasso regularization-path task, Dream-RSI reports up to about 162× fewer discovery-agent calls than SimpleTES in the project’s highlighted comparisons. On KernelBench-style work, they report fewer generations for comparable performance on some tasks and higher performance under similar budgets on others.

Why this is weird in a useful way

Agent labs often chase bigger models or longer tool loops. Dream-RSI treats exploration as programmable code sitting above an unchanged coding agent. Self-improvement becomes a meta-search over policies, paid for by replaying yesterday’s trees. That is recursive self-improvement without a weight update, and it reframes “world models” as something the agent already builds whenever it explores.

The paper does not claim dreaming invents branches the agent never tried. It claims the industry has been leaving free feedback on the table. If the results generalize, the next efficiency race in coding agents may be less about smarter generators and more about smarter reuse of the search history those generators already leave behind.