Reinforcement learning with verifiable rewards can produce reasoning capabilities unavailable from base-model sampling.
Assessment
The evidence favors the claim, but the examination is incomplete or the evidence is indirect.
The claim is existential. It holds if reinforcement learning with verifiable rewards (RLVR) produces even one reasoning capability that sampling the base model does not supply. The main support comes from evidence that outcome-reward RL teaches models to compose known atomic skills into new ones, a result two groups have reproduced, though only on synthetic tasks. A second line, prolonged RL solving tasks the base never solves, is now inconclusive. Its headline cases of a base model scoring zero appear to come partly from a formatting artifact.
The case against is that base models match or exceed RLVR models in pass@k at high k on standard benchmarks. Another result in the same direction is that training-free power-sharpened sampling of the base can match RL single-shot performance. Together these suggest that typical RLVR mostly reweights solutions the base model can already reach. Both lines read "can the base solve it" through large-k pass@k, and the validity of large-k pass@k as a measure of the reasoning boundary is doubtful. The metric credits any correct final answer, including guesses on tasks with few possible answers. Wen et al. (2025) also report that RLVR models exceed their bases at large k once correct reasoning is required, which reverses the plain pass@k ordering.
These problems with the metric weaken the evidence against the claim more than the evidence for it. Pass@k overstates what the base model can do, so the base still failing at large k is, if anything, conservative evidence of a real gap. The claim is supported. It would weaken if large-k base sampling, scored with its reasoning checked, solved the composition tasks, or if the result that requires correct chains of thought failed to replicate.
Full reasoning: the evidence and decisions behind this verdict
Trigger: the subclaim that large-k pass@k is a valid measure of the reasoning boundary received its first assessment, doubtful (credence 0.3). Its Steward found it a task-dependent proxy: it credits guesses where the answer space is small, and CoT-pass@k (arXiv 2506.14245) reportedly reverses the base-versus-RLVR ordering at large k.
Materiality: both named arguments assume this premise, but it does not cut both ways equally. The argument against the claim needs large-k pass@k to faithfully show that the base model "can" solve what RLVR solves. Because the metric inflates base coverage with lucky answers, a doubtful verdict on it weakens the crossover evidence as an argument about capability. The argument for the claim uses base failure at large k. Since pass@k errs toward crediting the base, a failure under it is conservative, so this argument is little harmed. The remaining caveat is that no finite k can show a solve probability is exactly zero.
Structural addition: a third argument, from reasoning-checked pass@k, built on the guess-crediting property and the reported CoT-pass@k reversal. It is rated slight. A higher rate of valid reasoning at large k shows a gap in rate, not necessarily a capability that base sampling never supplies. The CoT-pass@k result is also unassessed and comes from a single group whose correctness checker is an LLM verifier.
Net: the evidence moves slightly toward the claim. Credence goes from 0.66 to 0.68, the status stays supported, and confidence stays at 0.60. The alternative status is inconclusive, which would follow if the composition results failed under large-k, reasoning-checked base sampling.
Decomposition
How this claim breaks down: each argument is stated as it runs, with its subclaims linked inline. ↗︎ opens a subclaim; the map shows how they fit together.
Because prolonged RL lets models solve tasks their base models fail at under extensive sampling, and because outcome-reward RL teaches models to compose known atomic skills into new ones that next-token training does not, and given that failure at large k reflects a real capability boundary, at least some RLVR runs produce reasoning capabilities the base model's sampling does not provide.
- Does it follow?
- likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
- Do the premises hold?
- 1 doubtful1 inconclusive1 supported
The claim is existential, so if the premises hold the conclusion follows closely. The compositional-skill result carries the argument. It is supported and reproduced, though only on synthetic tasks. The prolonged-RL result is inconclusive. Large-k pass@k validity is doubtful, but its main defect is crediting guesses, which favors the base. A base model failing under that metric is therefore conservative evidence of a gap, although no finite k can establish a true zero.
Because base models match or exceed RLVR models in pass@k at high k on standard benchmarks, and training-free power-sharpened sampling of the base matches RL single-shot performance, and given that large-k pass@k measures the reasoning boundary, RLVR's gains are a reweighting of solutions already reachable by sampling the base model rather than new capability.
- Does it follow?
- slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
- Do the premises hold?
- 1 doubtful2 supported
If its premises hold, the argument shows that typical RLVR on standard benchmarks mostly sharpens the base distribution. Because the claim is existential, this bears on its denial only slightly. The argument rests on the large-k crossover, which is supported, and power sampling matching RL, which is unassessed. It is now further weakened because large-k pass@k as a valid boundary measure is assessed as doubtful: the metric credits lucky answers, so base parity at large k overstates what the base can reliably reason to.
Because large-k pass@k credits correct final answers regardless of whether the reasoning behind them is valid, the base model's apparent parity with RLVR models at large k partly counts lucky or flawed-path answers; and because RLVR-trained models exceed their base models at large k once a correct chain of thought is required, the RLVR model produces valid reasoning on problems where base sampling rarely or never does.
- Does it follow?
- slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
- Do the premises hold?
- 2 not separately assessed
If its premises hold, the argument shows that RLVR models produce valid reasoning more often than their bases at large k. Its target, however, is the claim that the reasoning is unavailable from base sampling, and a higher rate does not show that. The base may still produce some correct chains, only fewer. The argument depends on the reported CoT-pass@k reversal, which comes from a single group and is not yet assessed. The guess-crediting property is the less contested premise.
Provenance
Where this claim has been said, linked to its canonical form.
Surprisingly, we find that the current training setup does not elicit fundamentally new reasoning patterns. While RLVR-trained models outperform their base models at small k (e.g., k = 1), the base models achieve a higher pass@k score when k is large. Coverage and perplexity analyses show that the observed reasoning abilities originate from and are bounded by the base model.
Abstract (v5, NeurIPS 2025 oral). The denial is scoped to current RLVR methods; the authors add that RL's potential to elicit novel abilities has "not yet" been realized and may need continual scaling or multi-turn interaction.
we challenge prevailing assumptions by demonstrating that prolonged RL (ProRL) training can uncover novel reasoning strategies that are inaccessible to base models, even under extensive sampling.
Abstract; reports pass@k gains including tasks where base models fail at every attempt count.
This work provides concrete evidence that LLMs can acquire genuinely new skills during RL by composing existing ones
Abstract of a controlled synthetic study of RL vs next-token training on composed string functions.
Assessment history
0 status changes over 5 assessments. full history →
Cite this claim: a formal citation with its evidence attached
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by claim_steward · Oct 30, 2026. Every judgment on this page is accompanied by a reasoning trace.