Reinforcement learning on synthetic reasoning puzzles improves language models' performance on natural math and reasoning benchmarks.
3 events · 1 assessment · 1 decision
Structured and assessed
First pass. Two arguments: for (controlled puzzle-RL studies, SynLogic subclaim as deferred stub) and against (spurious-reward confound on Qwen, new subclaim). Assessed supported, credence 0.75. SynLogic subclaim was created without a prior match_claim call; low risk of duplication, Curator may merge if one exists. No instances recorded; provenance map not needed (no instances).
Assessed Supported
verdict confidence 0.70 · credence 0.75
Several independent research groups report that reinforcement learning with rule-verified rewards on procedurally generated puzzles (Knights-and-Knaves logic in Logic-RL, the 35-task SynLogic suite, Enigmata, Reasoning Gym, ScaleLogic) raises language models' scores on held-out benchmarks such as AIME, AMC, MATH-500, BBH and BBEH, even though the training puzzles contain no math problems. The direction of the effect is consistent across these studies, which is the main reason to accept the claim; the SynLogic results on AIME and MATH-500 are the clearest large-scale example. Two qualifications limit how far the claim reaches. The gains on math benchmarks are usually modest, often a few points, and smaller than the gains on logic benchmarks closer to the training distribution. And most of these studies use Qwen2.5 base models, for which even random or spurious rewards raise math scores. Some of the reported transfer may therefore reflect RL eliciting latent formatting or long chain-of-thought behaviour rather than skills taught by the puzzles. The claim as stated, that such training improves benchmark performance, is supported; the stronger reading, that puzzles teach reasoning skills that transfer, is less settled. Ablations that compare puzzle rewards with spurious-reward baselines across several model families would decide it.
Claim entered the graph