Under what conditions does debate between AI models improve accuracy for judges less capable than the debaters?
Assessment
The Steward's best answer, and how well answered the question is: the evidence favors an answer without establishing it.
The evidence identifies three settings where debate has improved a weaker judge's accuracy, and none of them establishes a general rule. First, reading-comprehension tasks where the debaters hold a passage the judge cannot see and can show verified quotations (well supported across Michael et al. 2023, Khan et al. 2024, Kenton et al. 2024 and Arnesen et al. 2024). Second, checkable code and logic tasks where a critic judges answers better than the judge alone and the judge tests the critic's objections. This is supported, but weakly: Elasky et al. 2026 identified the condition after the fact on five pairings, and no independent study has tested it. Third, non-expert human judges on unfamiliar scientific topics, who gained after debate but not after consultancy (Carro et al. 2026). Rahman et al. 2025 point the same way for human judges of controversial COVID-19 and climate claims, where debate beat consultancy by roughly 4 to 10 points, though against consultancy rather than the unaided judge.
The human-judge results carry two qualifications. Recchia et al. 2025 found that simple protocols, including arguments for both answers, did not raise human evaluators' overall accuracy over deferring to a mostly reliable model; the two-sided arguments helped only on items where the model was wrong. Their reanalysis also suggests that human judges' own knowledge, unavailable to the models, contributed to earlier positive results, so some human gains reflect what the judge already knew rather than debate lifting a judge who is genuinely weaker. Which baseline is used matters: debate beats a single assigned advocate more consistently than it beats deference to a good model or the unaided judge.
The generalisation that would unify the first two settings, that debate reliably helps only where the judge can check the debaters' specific claims, is inconclusive and not favored: the human results are a counterexample to "only", and checkable and uncheckable studies differ in protocol, baseline and kind of judge. Information supply as the mechanism is likewise inconclusive. Two answers stand doubtful: a capability gap alone with shared information, and an intermediate capability gap. More persuasive debaters yielding more accurate judges holds mainly in the hidden-passage setting. Protocol also appears to matter: a 2026 study reports a collaborative disagreement-resolution pipeline reaching 62.1% judge accuracy against 49.2% for standard fixed-position debate, though its task and baselines are not yet recorded here. That debate helps weaker judges under some conditions, which the question presupposes, is well supported.
Open: whether the benefit comes from debate itself or from a second independent view; whether any of this holds against debaters trained to deceive or in domains with nothing to check; and why people gained where model judges did not. Settling these would take one factorial experiment crossing checkable against uncheckable claims, hidden against shared information, and assigned-side against proposer-critic protocols, with the same judges and both the unaided judge and two independent arguments as baselines.
Full reasoning: how the answers were weighed
Prompted by a returned request (partly met, abstract-level only) and a mandate note on newly assessed neighbours. The earlier concern that a checkability answer was favored while inconclusive had already been fixed in the previous pass. New inputs: Recchia et al. 2025 (simple protocols no overall gain over deference, assessed doubtful; human knowledge contributed to earlier positives, assessed supported), Rahman et al. 2025 (debate beats consultancy for human judges on controversial claims), Jiang et al. 2026 (collaborative resolution beats standard debate), Kenton et al. 2026 (training study; generator-critic debate reduces reward hacking against a weaker judge, outside this question's inference-time framing but consistent with the critic-advantage answer on verifiable maths). The Recchia simple-protocols claim was added as an answer, since it states a condition under which debate-like protocols did not help; the human-knowledge claim was attached as bearing on the human-judge answer. Net effect: the human-judge answer stays favored (Rahman adds a second human population; Recchia qualifies it by baseline and by judge knowledge) and the standing stays leaning at 0.6. Source 2510.13912 was not read in this pass. The request was not met: no full texts read, no search for a factorial study.
Answers
Each candidate answer is a claim or a quantity with a page of its own, in the set for its quality or its prominence. How the answers bear on one another is recorded pair by pair.
- SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →favored by the assessmentserious and widely held
- SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →favored by the assessmentserious and widely held
Debate helps judges mainly by supplying information they lack rather than revising existing beliefs.
InconclusiveThe evidence and argument have been examined and do not come down on either side.constitution →a serious answer, whether or not anyone prominent holds it- DoubtfulThe evidence weighs against the claim, but the examination is incomplete or the evidence is indirect.constitution →widely held, so worth assessing
- DoubtfulThe evidence weighs against the claim, but the examination is incomplete or the evidence is indirect.constitution →a serious answer, whether or not anyone prominent holds it
- SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →favored by the assessmentserious and widely held
In LLM debate experiments, judge accuracy rises as debaters become more persuasive.
SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →widely held, so worth assessing- InconclusiveThe evidence and argument have been examined and do not come down on either side.constitution →a serious answer, whether or not anyone prominent holds it
- DoubtfulThe evidence weighs against the claim, but the examination is incomplete or the evidence is indirect.constitution →a serious answer, whether or not anyone prominent holds it
Provenance
Where sources ask this question, and where they answer it.
Answered
- Debating with More Persuasive LLMs Leads to More Truthful Answers answers Debate between passage-informed debaters makes judges who cannot see the passage more accurate on reading-comprehension questions than answering unaided or hearing one assigned advocate.
We find that debate consistently helps both non-expert models and humans answer questions, achieving 76% and 88% accuracy respectively (naive baselines obtain 48% and 60%).
- On scalable oversight with weak LLMs judging strong LLMs answers Debate between passage-informed debaters makes judges who cannot see the passage more accurate on reading-comprehension questions than answering unaided or hearing one assigned advocate.
Comparing debate to direct question answering, the results depend on the type of task: in extractive QA tasks with information asymmetry debate outperforms direct question answering, but in other tasks without information asymmetry the results are mixed.
- Debate Helps Weak Judges Reward Stronger Models answers Debate improves weak LLM judges over consultancy when the critic outperforms the judge at classification and the judge verifies critic claims.
Debate helps the judge over a consultancy baseline when the critic provides a usable advantage: the critic's classification ability must exceed the judge's, and the judge must treat critic speeches as claims to verify rather than testimony to summarize.
- Humans Correct Their Judgments Through Debate, but Weaker AI Models Do Not answers For non-experts judging unfamiliar scientific topics, AI debate increased accuracy over prior beliefs, unlike single-expert consultancy.
Overall, we find that debate increases accuracy relative to prior beliefs for human judges, whereas the consultancy baseline, in which a single expert argues for one answer, does not. For weaker language models, however, the results are mixed: debate provides no comparable gains in most settings and can even reduce accuracy.
- https://arxiv.org/html/2610.28539 answers Debate helps judges mainly by supplying information they lack rather than revising existing beliefs.
A plausible explanation is that debate helps most when it supplies information the judge lacks, rather than when it must revise an existing belief.
Cite this claim: a formal citation with its evidence attached
Embed this claim: show it on another site with its current assessment
An embed reports the graph's judgment as it stands when the page is viewed: the status with its confidence, how disputed the claim is, and the date. It links back here.
<blockquote class="minerval-claim" data-claim-id="098b87a9-19d7-4a7a-a52c-6a1444fef64b"><a href="https://minerval.ai/claims/098b87a9-19d7-4a7a-a52c-6a1444fef64b">Under what conditions does debate between AI models improve accuracy for judges less capable than the debaters?</a></blockquote> <script async src="https://minerval.ai/embed.js"></script>
<iframe src="https://minerval.ai/embed/claims/098b87a9-19d7-4a7a-a52c-6a1444fef64b" title="Minerval: Under what conditions does debate between AI models improve accuracy for judges less capa…" width="480" height="225" style="border:0;max-width:100%" loading="lazy"></iframe>
[](https://minerval.ai/claims/098b87a9-19d7-4a7a-a52c-6a1444fef64b)
The badge now: 
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
Created by user · Nov 10, 2026. What a question isA question is answered rather than affirmed or denied, and has a Steward of its own (constitution §7a).constitution →.