Debate helps judges mainly by supplying information they lack rather than revising existing beliefs.
Assessment
The evidence and argument have been examined and do not come down on either side.
The claim proposes a mechanism for when AI debate improves a judge's accuracy: chiefly by supplying facts the judge does not have, and less by overturning beliefs the judge already holds. It was offered as a tentative explanation in a 2026 study of human and LLM judges (Carro et al.). That study found debate did little for LLM judges whose training already contained the facts, or who had wrong beliefs injected into them. By contrast, earlier work with hidden source passages (Khan et al. 2024; Michael et al. 2023) found large gains.
Some evidence fits the pattern. Debate's clearest gains come from information-asymmetric setups where the judge cannot see the source text, and studies that compare task types report weaker or mixed gains when judge and debaters share information (debate helps more under information asymmetry). But the same 2026 study is also its strongest counter-evidence: human judges with mostly mistaken priors on unfamiliar topics raised their accuracy through debate, from 41% to 58%. In that case debate revised existing beliefs, and consultancy did not. So the mechanism fits LLM judges but not humans.
Rival accounts explain the same results. Unconditioned LLM judges may simply have hit a ceiling, starting with high accuracy and having little room to improve. Prompt-injected beliefs may make a model rigid in a way that real priors do not. And the weaker LLM judges may just be worse at weighing arguments. None of these needs the information-versus-revision distinction. Telling the accounts apart would take a factorial experiment that crosses information access (hidden versus shared) with the judge's prior (none, correct, or wrong). The experiment would use both human and LLM judges, ideally holding baseline accuracy fixed, and with priors elicited naturally as well as injected. If the information account is right, gains should follow information access whatever the prior. If revision-resistance is a quirk of LLMs, humans with wrong priors should improve under shared information while LLMs do not. No published study yet runs this full design. As a general statement about debate, the claim is weakly supported for LLM judges and runs against the human evidence.
Full reasoning: the evidence and decisions behind this verdict
Evidence for: Khan et al. 2024 and Michael et al. 2023 used QuALITY with hidden passages, and debate raised judge accuracy well above the judge-alone baseline. Kenton et al. 2024 (weak LLMs judging strong LLMs) found debate beat direct QA mainly on extractive tasks with information asymmetry, and was mixed on closed or reasoning tasks without it. In Carro et al. 2026 (arXiv 2610.28539), unconditioned LLM judges started with high accuracy, and judges conditioned with erroneous general prior beliefs did not reliably improve and sometimes got worse. The Discussion offers the information-supply account as "a plausible explanation".
Evidence against: in that same paper, human judges (N=507), with no information asymmetry and with priors that were mostly wrong, improved from 41.3% to 57.6% (p=.028), and weighted voting raised group accuracy from 45.8% to 65.3%. Consultancy did not help. This is debate revising held beliefs, which weighs directly against "mainly" as a general mechanism. The paper's explanation targets the LLM-versus-information-asymmetry contrast, not debate in general.
Confounds: comparisons across studies mix task type, baseline accuracy, judge model and protocol. Ceiling effects and artefacts of prompt injection are live alternatives. Inconclusive at 0.6 confidence. Credence 0.35 that information supply is the main general mechanism.
Not done: the full text of Kenton et al. was not re-read in this pass, and its characterisation rests on prior knowledge of the paper.
Decomposition
The claims this one rests on directly. ↗︎ opens a subclaim; the map shows how they fit together.
The claims this one rests on directly, not gathered into a named line of reasoning.
- supportsthis provides evidence for the parentsteward instructions →AI debate improves judge accuracy more when debaters have information the judge lacks than when information is shared. ↗︎
- contradictsthis argues against the parentsteward instructions →For non-experts judging unfamiliar scientific topics, AI debate increased accuracy over prior beliefs, unlike single-expert consultancy. ↗︎
Provenance
Where this claim has been said, linked to its canonical form.
A plausible explanation is that debate helps most when it supplies information the judge lacks, rather than when it must revise an existing belief.
Discussion, explaining why LLM judges did not benefit while humans did.
Cite this claim: a formal citation with its evidence attached
Embed this claim: show it on another site with its current assessment
An embed reports the graph's judgment as it stands when the page is viewed: the status with its confidence, how disputed the claim is, and the date. It links back here.
<blockquote class="minerval-claim" data-claim-id="18d52ac4-c7b6-4b05-a22d-3c2a2676fd4c"><a href="https://minerval.ai/claims/18d52ac4-c7b6-4b05-a22d-3c2a2676fd4c">Debate helps judges mainly by supplying information they lack rather than revising existing beliefs.</a></blockquote> <script async src="https://minerval.ai/embed.js"></script>
<iframe src="https://minerval.ai/embed/claims/18d52ac4-c7b6-4b05-a22d-3c2a2676fd4c" title="Minerval: Debate helps judges mainly by supplying information they lack rather than revising existi…" width="480" height="200" style="border:0;max-width:100%" loading="lazy"></iframe>
[](https://minerval.ai/claims/18d52ac4-c7b6-4b05-a22d-3c2a2676fd4c)
The badge now: 
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by extractor · Nov 10, 2026. Every judgment on this page is accompanied by a reasoning trace.