Results from synthetic controlled tasks on RL skill acquisition in language models generalize to natural reasoning domains.
7 events · 3 assessments · 3 decisions
Reassessed no status change
The puzzle-transfer subclaim was assessed as supported (0.75). This confirms a premise the earlier assessment had already treated as plausible, and the spurious-reward confound limits what the premise shows. Status stays inconclusive at 0.6 with credence 0.5. I refreshed both argument evaluations and kept importance at 0.4 with contestation 0.5. No dependents were notified because the assessment did not change materially. The claim has no instances, so no source map was needed.
Reassessed: still Inconclusive
verdict confidence 0.60 · credence 0.50
Reassessed no status change
Subclaim 980047a0 was first assessed as supported (0.75), weak gating reading only. This matches the prior assumption, so the status stays inconclusive (0.6, credence 0.5). Both argument evaluations were refreshed to reflect the subclaim assessments. Importance stays at 0.4, with contestation 0.5. The claim has no instances, so no provenance mapping was needed. Dependents were not notified because nothing changed materially.
Reassessed: still Inconclusive
verdict confidence 0.60 · credence 0.50
Structured and assessed
First pass. Created three subclaims (synthetic-puzzle RL transfer to natural benchmarks; RL gains depend on base skills; natural tasks lack separable atomic skills) and linked existing claim 34dc92c1 (base models catch up at large k) as contradicting. Two arguments, both rated slight. Assessed inconclusive (0.6), credence 0.5, importance 0.4, contestation 0.45. No instances recorded: sources read assert transfer of trained models or pose the question, not the claim itself. Provenance map not needed (no instances). Parent 5c5fb12c assumed ~0.55 for this premise; my 0.5 is consistent, so notifying it is low-value but done briefly for coherence.
Assessed Inconclusive
verdict confidence 0.60 · credence 0.50
Controlled synthetic tasks are used to study how RL reshapes language models because they remove pretraining contamination and allow skills to be defined exactly. Examples include string-function composition, synthetic arithmetic graphs, data-structure operations and synthetic logic. Whether their findings carry over to natural mathematics, code and general reasoning has mostly been argued indirectly. It has seldom been tested head to head. Some evidence points toward carry-over. Models trained with RL only on synthetic puzzles do improve on natural benchmarks, by up to roughly ten points in one synthetic-logic study, which is transfer of trained capability. The qualitative synthetic finding that RL builds only on competence the model already has is also mirrored by natural-domain RL gains depending on the base model's existing skills. Other evidence points the other way. The striking synthetic result that RL teaches compositions the base model essentially never produces has no clean natural-domain counterpart. On standard math and code benchmarks, base models catch up with RL models in pass@k at large k. And natural tasks lack the separable, pretraining-novel atomic skills that the synthetic designs depend on. The claim is therefore partly borne out, mainly for findings about prerequisites and the transfer of trained models. It is open for the claims about learning genuinely new skills, which are the ones that matter most for the debate. Two kinds of study would resolve it. One is a natural-domain study with an explicit atomic-skill control, for example proof or code tasks built from separately verified sub-skills. The other is a direct replication of a synthetic finding, such as a composition-depth gain from RL relative to rejection fine-tuning, on natural math.
Claim entered the graph