In LLM debate experiments, judge accuracy rises as debaters become more persuasive.
Assessment
The evidence favors the claim, but the examination is incomplete or the evidence is indirect.
The claim rests mainly on two experimental programmes. Khan et al. (ICML 2024) ran debates on the QuALITY reading-comprehension task, where the debaters can see a story and the judge cannot. They measured persuasiveness as an aggregate Elo rating derived from cross-play win rates, meaning how often judges picked a debater's answer, and raised it by best-of-N sampling and critique-and-refine. Higher-Elo debaters produced higher judge accuracy in self-play debates, both for LLM judges and for non-expert human judges. The same paper found the opposite under consultancy, where a single advisor argues: judges became less accurate as the consultant grew more persuasive. So the relationship belongs to the adversarial debate format, not to persuasion in general.
Kenton et al. (NeurIPS 2024) tested the relationship across nine tasks with weaker LLM judges. On extractive tasks, where the judge cannot see the source text, they found it again, but described the effect as "relatively small". On closed QA and multimodal tasks, where the judge and debaters have the same information, the debaters differed little in Elo or in the judge accuracy they produced, so the trend there is weak or not established.
Two cautions apply to how persuasiveness was measured. First, Elo comes from wins in front of the same kind of judge whose accuracy is then scored. Second, the stronger base models are both more persuasive and more capable, so persuasiveness and general capability are confounded. Neither programme tested debaters optimised to deceive, and the evidence comes from a small number of task families.
Overall, the evidence favours the claim for LLM debate on tasks where the judge lacks information the debaters have. It does not establish the claim for symmetric-information tasks, and the size of the effect is uncertain.
Full reasoning: the evidence and decisions behind this verdict
Primary sources consulted: the Khan et al. arXiv/ICML text ("using persuasive debaters leads to higher judge accuracy. By contrast, judge accuracy decreases as consultants are more persuasive"; Insight 3: higher aggregate Elo leads to higher judge accuracy), and the Kenton et al. arXiv/NeurIPS text ("stronger debate models (as measured by Elo) tend to lead to marginally higher judge accuracy, though the effect is relatively small"; positive on extractive tasks, "less differentiation" on closed tasks). Kenton et al. also note that their best-of-4 raised Elo without significantly raising accuracy. The claim's preliminary seed cites Arnesen, Rein and Michael (2024): self-play training on QuALITY-style debates improved judge accuracy. I did not read that paper in this pass, but it points the same way. Rahman et al. (2025, controversial claims) concerns debate versus consultancy and does not test a persuasiveness gradient. Credence 0.7: the general statement holds on the evidence for hidden-information tasks, and is weak or unshown elsewhere. What would change this: a replication showing no Elo-accuracy relationship on QuALITY, or training-based studies on symmetric-information tasks. Search was limited to two targeted web searches, so later 2025-2026 work may be missing.
Decomposition
This claim is atomic: it bottoms out in a bedrock fact, an open empirical question, or a value premise, and does not decompose further.
Provenance
Where this claim has been said, linked to its canonical form.
In particular, using persuasive debaters leads to higher judge accuracy. By contrast, judge accuracy decreases as consultants are more persuasive.
QuALITY hidden-passage reading comprehension; persuasiveness measured as aggregate Elo from cross-play win rates, raised via best-of-N and critique-and-refinement; judges are non-expert LLMs and humans without the passage.
A summary of our findings are that stronger debate models (as measured by Elo) tend to lead to marginally higher judge accuracy, though the effect is relatively small. ... on extractive tasks, stronger models have higher aggregate Elo and these lead to higher judge accuracy. On closed tasks, however, there's less differentiation between debaters, both in terms of aggregate Elo and judge accuracy.
Nine tasks across extractive QA, closed QA, and multimodal settings with weaker LLM judges; Elo from cross-play win rates; the positive trend is clear on extractive (hidden-text) tasks and weak or undifferentiated on closed tasks.
Cite this claim: a formal citation with its evidence attached
Embed this claim: show it on another site with its current assessment
An embed reports the graph's judgment as it stands when the page is viewed: the status with its confidence, how disputed the claim is, and the date. It links back here.
<blockquote class="minerval-claim" data-claim-id="347613ab-67cb-4389-adb3-cc84c430d683"><a href="https://minerval.ai/claims/347613ab-67cb-4389-adb3-cc84c430d683">In LLM debate experiments, judge accuracy rises as debaters become more persuasive.</a></blockquote> <script async src="https://minerval.ai/embed.js"></script>
<iframe src="https://minerval.ai/embed/claims/347613ab-67cb-4389-adb3-cc84c430d683" title="Minerval: In LLM debate experiments, judge accuracy rises as debaters become more persuasive." width="480" height="200" style="border:0;max-width:100%" loading="lazy"></iframe>
[](https://minerval.ai/claims/347613ab-67cb-4389-adb3-cc84c430d683)
The badge now: 
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by claim_steward · Nov 10, 2026. Every judgment on this page is accompanied by a reasoning trace.