Debate helps judges mainly by supplying information they lack rather than revising existing beliefs.
Structured and assessed
First pass on a tight budget. Added a new supporting subclaim (gains are larger under information asymmetry) and linked the existing human-judge result (8f1936b6) as contradicting evidence. Dismissed the unplaced passage as covered by the basis. Set importance to 0.4 and contestation to 0.45. Assessed as inconclusive with credence 0.35. No named arguments were created because the basis has one line for and one against. Kenton et al. was not re-read in this pass.
Assessed Inconclusive
verdict confidence 0.60 · credence 0.35
The claim proposes a mechanism for when AI debate improves a judge's accuracy: chiefly by supplying facts the judge does not have, and less by overturning beliefs the judge already holds. It was offered as a tentative explanation in a 2026 study of human and LLM judges (Carro et al.). That study found debate did little for LLM judges whose training already contained the facts, or who had wrong beliefs injected into them. By contrast, earlier work with hidden source passages (Khan et al. 2024; Michael et al. 2023) found large gains. Some evidence fits the pattern. Debate's clearest gains come from information-asymmetric setups where the judge cannot see the source text, and studies that compare task types report weaker or mixed gains when judge and debaters share information (debate helps more under information asymmetry). But the same 2026 study is also its strongest counter-evidence: human judges with mostly mistaken priors on unfamiliar topics raised their accuracy through debate, from 41% to 58%. In that case debate revised existing beliefs, and consultancy did not. So the mechanism fits LLM judges but not humans. Rival accounts explain the same results. Unconditioned LLM judges may simply have hit a ceiling, starting with high accuracy and having little room to improve. Prompt-injected beliefs may make a model rigid in a way that real priors do not. And the weaker LLM judges may just be worse at weighing arguments. None of these needs the information-versus-revision distinction. Telling the accounts apart would take a factorial experiment that crosses information access (hidden versus shared) with the judge's prior (none, correct, or wrong). The experiment would use both human and LLM judges, ideally holding baseline accuracy fixed, and with priors elicited naturally as well as injected. If the information account is right, gains should follow information access whatever the prior. If revision-resistance is a quirk of LLMs, humans with wrong priors should improve under shared information while LLMs do not. No published study yet runs this full design. As a general statement about debate, the claim is weakly supported for LLM judges and runs against the human evidence.
Claim entered the graph