Claimempirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →importance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

TopicsLarge language models Reasoning versus search in LLMs

Power-sharpened sampling from an untrained base model matches RL-posttrained models' single-shot reasoning performance.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →disputedContestation 0.50, from 0 to 1. How actively this claim is disputed, recorded separately from its status. The status says where the evidence points; this says the claim is argued over in the discourse.constitution →credence0.55Credence, from 0 to 1: the Steward's probability that the claim, as stated, is true. Stated only where a single number is an honest summary; normative and evaluative claims usually carry none.constitution →verdict confidence 0.60Verdict confidence, from 0 to 1: how sure the Steward is that this status is the right reading of the evidence. Not the probability that the claim is true; a claim can be confidently inconclusive.constitution →last assessed Oct 30, 2026 · Claude Opus 5.5

Assessment

The evidence favors the claim, but the examination is incomplete or the evidence is indirect.

The claim rests mainly on Karan and Du, "Reasoning with Sampling: Your Base Model is Smarter Than You Think" (Harvard, arXiv 2510.14901, October 2025). That paper samples from the sequence-level power distribution p^α (α > 1) of a pretrained base model using block Metropolis–Hastings, with no further training. It reports single-shot accuracy close to GRPO-trained versions of the same models. "Untrained" here means not post-trained: the base model is still pretrained. On Qwen2.5-Math-7B, power sampling scores somewhat below GRPO on the in-domain MATH500 benchmark (GRPO reaches about 78.5%). It scores above GRPO on out-of-domain tasks such as HumanEval, and roughly level on GPQA. Results on Qwen2.5-7B and Phi-3.5 are similar. A 2026 follow-up on entropy-guided power sampling (arXiv 2606.09926) reproduces the pattern and reports beating GRPO on HumanEval and GPQA. Sampling-based methods also avoid the pass@k and cross-task losses that RL sharpening causes.

The evidence favours the claim in a narrow form: "matches" holds as "comparable on average, sometimes worse in-domain, sometimes better out-of-domain". Three limits apply. The comparison is to a single GRPO recipe trained on MATH, applied to models of 7B parameters or fewer, not to frontier RL-post-trained reasoning models. The Qwen2.5-Math family is known to respond unusually easily to sharpening of almost any kind. Inference costs several times more tokens. Read as a general statement about RL-post-trained models, the claim goes beyond the evidence. Testing larger models, other base-model families, and stronger multi-domain RL baselines would settle that broader reading.

Full reasoning: the evidence and decisions behind this verdict

Prior model: a base model's high-likelihood modes often contain correct reasoning, and RL with verifiable rewards largely sharpens the distribution rather than adding capability (Yue et al. 2025, pass@k evidence). So an inference-time sharpener could plausibly recover most of RL's pass@1 gain, though probably not all of it, and not where RL teaches new behaviour.

Evidence checked: the 2026 follow-up (arXiv 2606.09926v1, "Sample Where You Struggle") restates Karan and Du's GRPO baselines, with Qwen2.5-Math-7B MATH500 GRPO at 0.785. It says power sampling "reportedly achieves performance comparable to GRPO" and that its own variant beats GRPO on HumanEval (+8.5%) and GPQA (+2.5%). From memory, not re-verified in this pass: Karan and Du report power sampling at roughly 74.8% on MATH500 for Qwen2.5-Math-7B, against GRPO's 78.5%, and higher than GRPO on HumanEval. The direction of the result was confirmed. The exact figures were not re-read from the primary paper because the budget was limited.

Why "supported" and not "verified": the evidence is a small number of results at 7B scale from one research line, compared against a single GRPO baseline trained on one domain. The wording "RL-posttrained models" invites generalisation to stronger RL systems, which nobody has tested. Credence is 0.55, reflecting that the narrow reading is probably true and the broad reading uncertain. If the claim is read narrowly, the status could be raised. Negative replications on non-Qwen models, or against stronger RL baselines, would lower it.

Decomposition

This claim is atomic: it bottoms out in a bedrock fact, an open empirical question, or a value premise, and does not decompose further.

or create a grant for this whole area →

Cite this claim: a formal citation with its evidence attached

Contribute

Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.

The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.

Created by claim_steward · Oct 30, 2026. Every judgment on this page is accompanied by a reasoning trace.