Logging a failed move made small LLM players likelier to retry it, a Passau study finds

Writing a failed move back into a small language model's transcript, the standard way game and agent harnesses report an error, can make the model more likely to try that same move again. That is the main result of a preprint posted to arXiv on October 21 by Esmail Gumaan of the University of Passau, who tested eight open models between 135 million and 1.7 billion parameters on Sokoban, Connect Four and Othello.

The catch is that the effect runs in both directions. Moves a model would not normally have picked became more likely after the failure was recorded, while moves it was already inclined to play became less likely, though greedy decoding still repeated 38% of those.

The move string is the problem

The study builds probe positions by nudging a correct move into a failing one, such as an adjacent column in Connect Four or a wall in Sokoban. It then measures how the failed move's probability changes once the harness logs the move plus a referee message like "that column is already full."

Pooled across the eight models, the record raised the failed move's probability by 1.38 nats per token, a log-probability measure. Almost all of that came from the move simply appearing in the transcript. Swapping the error message for a neutral "Move received." changed the result by only 0.04 nats, an amount the paper cannot distinguish from zero.

It was not universal. Four models inverted clearly, with Qwen3-0.6B showing the largest effect. Llama-3.2-1B did the opposite: seeing its own move made it less likely to repeat it. The effect was clear for illegal moves and much weaker for legal moves that lose. On items built from each model's own top choices, the sign flipped and the record helped.

Fixes that do and do not work

Adding "do not repeat" to the system prompt did nothing measurable. A clean retry, where the harness wipes the attempt and asks again, was the worst option for repetition: in short free play games, 94% of failed moves under clean retry repeated an earlier failure exactly, against 72% with the standard record.

Describing the failure without writing the move string removed about 76% of the inversion on average, but it made things worse for two models. Banning failed strings at the decoder stopped exact repeats, yet the share of illegal moves barely moved, from 81% to 79%. The models simply swapped one illegal move for another.

Tracing one model family, OLMo 2 1B, through its training stages, the paper finds the inversion already present in the base model and mostly removed at the DPO preference tuning stage.

How far it goes

Every number comes from small models run in float32 on a single free Colab GPU. A planned 7 billion parameter check did not fit in memory, and the paper's reinforcement learning experiment was not run. The two models used in free play were weak players, with 81% of their moves rejected.

The author recommends that harnesses keep the failed string out of the error message and pair any rewording with a decoder constraint. Whether larger models show the same pattern is still untested.