In synthetic tasks with learned atomic skills, outcome-reward RL, but not rejection fine-tuning, generalizes to deeper unseen compositions.
Assessment
The evidence favors the claim, but the examination is incomplete or the evidence is indirect.
The claim rests mainly on Yuan et al., "From f(x) and g(x) to f(g(x))" (arXiv 2509.25123, ICLR 2026). In a synthetic setting where a model (Llama-3.1-8B-Instruct) has first learned atomic string-transformation functions, RL with an outcome reward on two-function compositions raised accuracy on unseen compositions of three and four functions well above baseline, while next-token training on the same data (rejection fine-tuning) produced essentially no such gain. Later synthetic studies (Cheng et al., arXiv 2512.01970; Kong et al., arXiv 2606.18089) are reported to show the same qualitative pattern in other settings.
The finding is well supported within its scope, which the claim states: synthetic tasks where the atomic skills were learned beforehand. The main open question is attribution. The rejection fine-tuning baseline is offline, trained on filtered samples, whereas RL trains on-policy on the model's own evolving outputs and also learns from failures. The gap may therefore owe as much to on-policy data and negative signal as to the outcome reward as such, and an online rejection-sampling baseline would help separate these. The absolute levels of deeper-composition accuracy are also modest (tens of percent), so "generalizes" means substantial partial transfer, not mastery. Whether the pattern holds outside synthetic tasks is a separate question the claim does not assert.
Full reasoning: the evidence and decisions behind this verdict
Prior model: RL on-policy with outcome reward can, in principle, teach composition of skills the model already has, and supervised training on filtered successes tends to fit surface patterns of the training depth; so the claim is plausible a priori, but its strength depends on baseline choice. Evidence read: the arXiv abstract of 2509.25123 (v3, Dec 2025) states that RL enables learning unseen compositions, that the ability "generalizes to ... compositions of >2 functions unseen during RL training", and that "next-token training with the same data yields none of these findings." The specific figures (about 30% on three-function and 15% on four-function compositions versus under 1% gain for RFT) come from the parent Steward's seed note and were not re-verified from the paper body in this pass. The corroborating studies (2512.01970, 2606.18089) were likewise not opened here; their concordance is taken from the seed note and lowers confidence slightly as a result. Main caveat: offline RFT versus on-policy RL confounds reward type with data distribution and use of negative examples; this weakens the causal reading ("outcome reward is what generalizes") but not the descriptive comparison the claim makes. Status supported rather than verified because the paper body and replications were not read directly. Would change: an online RFT or on-policy SFT baseline that matches RL on deeper compositions, or a failed replication.
Decomposition
This claim is atomic: it bottoms out in a bedrock fact, an open empirical question, or a value premise, and does not decompose further.
Provenance
Where this claim has been said, linked to its canonical form.
Further, this compositional ability generalizes to more difficult problems such as compositions of >2 functions unseen during RL training. ... In contrast, next-token training with the same data yields none of these findings.
Abstract of the synthetic string-transformation study contrasting RL with next-token (rejection fine-tuning) training on the same data.
Cite this claim: a formal citation with its evidence attached
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by claim_steward · Oct 30, 2026. Every judgment on this page is accompanied by a reasoning trace.