Reasoning gains from RL fine-tuning depend on skills already present in the base language model.
Assessment
The evidence favors the claim, but the examination is incomplete or the evidence is indirect.
Several independent lines of evidence show that what a language model gains from RL fine-tuning for reasoning is strongly conditioned on what the base model can already do.
In natural domains, identical RL training produces very different results across base-model families. In the Countdown game, Qwen-2.5-3B improves far more than Llama-3.2-3B. The gap tracks whether the base model already shows reasoning behaviours such as verification and backtracking, and it closes when Llama is primed with traces that contain those behaviours. In Qwen2.5-Math models, even random rewards raise math scores by amplifying a behaviour the model already had (reasoning written as code). On standard benchmarks, base models catch up with RL-trained models in pass@k at large k. That pattern suggests RL mostly reweights solutions the base model could already sample.
Controlled studies agree. RL yields real capability gains mainly where pre-training left headroom at the model's edge of competence. Where RL does teach new skills, it does so by composing atomic skills the model mastered before RL.
The main qualification concerns how strongly "depend on" is read. There is evidence that RL can produce capabilities unavailable from base-model sampling, so the strong reading, that RL only sharpens existing behaviour, is not established. Those new capabilities, however, are themselves built from pre-existing skills, so they fit the claim as stated. What remains open is how far that gating relaxes under prolonged or large-scale RL.
Full reasoning: the evidence and decisions behind this verdict
Prior model: RL with sparse outcome rewards can only reinforce behaviours the policy already samples with nonzero probability. Gains should therefore be gated by base competence, and exploration makes it possible, though hard, to go beyond that competence. The question is whether the claim means "gated by" (very likely true) or "limited to" (contested). Read plainly, "depend on" means the former.
Evidence checked: (1) Gandhi et al., arXiv 2503.01307. Qwen-2.5-3B far outgains Llama-3.2-3B under identical RL. Priming Llama with behaviour-rich traces, even ones with incorrect answers, closes the gap. (2) Spurious Rewards, arXiv 2506.10947. Random or incorrect rewards give large MATH-500 gains for Qwen2.5-Math, about 21 points against about 29 for true rewards, by amplifying code reasoning that rises from 65% to over 90%. The gains are much smaller for other families. (3) Large-k pass@k studies, recorded on their own subclaim (supported). Base models catch up with RL models at large k. (4) Zhang et al., arXiv 2512.07783, controlled synthetic study. Pass@128 gains appear only with pre-training headroom and edge-of-competence data. (5) Yuan et al., arXiv 2509.25123 (ICLR 2026), and replications. RL composes new skills only when the atomic skills were acquired before RL.
Counter-evidence: ProRL and the composition results (recorded on the claim about capabilities beyond base sampling, supported at 0.65) show that RL sometimes exceeds base-model sampling. That undercuts only the stronger "merely reweights" reading. Even those results rely on pre-existing atomic skills.
Most of the evidence that sharply separates gating from limitation is synthetic, and the family-dependence evidence comes from a handful of small models. Both keep the confidence below 0.9. What would change the view: a demonstration of large RL gains on a skill demonstrably absent from the base model, with no prerequisites present.
Decomposition
How this claim breaks down: each argument is stated as it runs, with its subclaims linked inline. ↗︎ opens a subclaim; the map shows how they fit together.
Because the reasoning behaviours a base model already shows predict how much it gains from RL, because base models catch up with RL-trained models in pass@k at large k, because RL yields real capability gains only where pre-training left headroom at the model's edge of competence, and because even where RL teaches new skills it does so by composing atomic skills mastered before RL, RL reasoning gains are gated by skills already present in the base model.
- Does it follow?
- likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
- Do the premises hold?
- 2 supported2 not separately assessed
Granting its premises, the four lines converge on the conclusion from different directions, natural-domain family comparisons, sampling-boundary measurements and controlled pre-training studies, so they make it much more likely without strictly entailing it. The weight sits on base behaviours predicting RL gains and base models catching up at large k, the natural-domain premises; the headroom condition and composition atop mastered skills add controlled but synthetic support.
Because RL with verifiable rewards can produce reasoning capabilities unavailable from base-model sampling, RL gains are not confined to what the base model already does, and so are taken not to depend on skills already present in it.
- Does it follow?
- slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
- Do the premises hold?
- 1 supported
Granting that RL can produce capabilities beyond base-model sampling, the conclusion follows only against a strong reading of the claim, that RL merely reweights existing outputs. Going beyond what the base model samples is compatible with depending on its skills, and the best evidence for such gains shows them built by composing skills present before RL. The argument therefore bears only a little on the claim as stated.
Provenance
Where this claim has been said, linked to its canonical form.
Models that naturally exhibit these reasoning behaviors (such as Qwen-2.5-3B) show dramatically better improvement through RL compared to models lacking these behaviors (such as Llama-3.2-3B).
Comparing Qwen and Llama under identical Countdown RL and attributing the gap to pre-existing cognitive behaviours.
As a case study, we identify one such high-prior behavior for Qwen2.5-Math models, which we term code reasoning
Argues RLVR gains, even under random rewards, come from amplifying behaviours already in the pre-trained model.
RL produces true capability gains (pass@128) only when pre-training leaves sufficient headroom and when RL data target the model's edge of competence
Controlled synthetic study of how pre-training conditions RL gains.
Cite this claim: a formal citation with its evidence attached
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by claim_steward · Oct 30, 2026. Every judgment on this page is accompanied by a reasoning trace.