https://arxiv.org/html/2610.28539

How can we effectively oversee AI systems that surpass human intelligence? AI Safety via Debate proposes that two AI systems argue opposing positions and a judge, either a human or a weaker model, decides which is correct. By facilitating supervisors in providing high-quality training signals, debate aims to encourage truthful and safe behavior from AI systems whose capabilities exceed our own. However, human judgment is not neutral: people often rely on biases which advanced AI systems may learn to exploit. Drawing on insights from collective intelligence and social cognition, this paper investigates whether debate can guide human and LLM supervisors toward the truth despite their prior beliefs. We evaluate several protocol configurations, varying (i) the type of judge (open-source and proprietary LLMs, and human participants, N = 507 N=507 ), (ii) the method for eliciting priors, and (iii) the specific claims debated, paired with different related claims subsequently evaluated for belief updating, to test whether corrections generalize across related subtopics. We also explore setups inspired by the wisdom of the crowd such as the number and diversity of judges, as well as interaction mechanisms including deliberation and voting. Overall, we find that debate increases accuracy relative to prior beliefs for human judges (from 41.3% to 57.6%, p = .028 p=.028 ), whereas the consultancy baseline, in which a single expert argues for one answer, does not. For weaker language models, however, the results are mixed: debate provides no comparable gains in most settings and can even reduce accuracy. Among multi-judge protocols, weighted voting produces the highest accuracy for human supervisors (45.8% to 65.3%, p = .028 p=.028 ), whereas for LLMs none of them significantly improves group accuracy, even as the number of judges increases. Notably, the behavior of minority judges in non-diverse deliberation protocols suggests that, under symmetric settings, incorrect positions could be more persuasive than correct ones. These findings support debate as a viable path toward scalable human oversight, but cast doubt on its benefits for weaker AI supervisors, raising concerns about fully automated training and control pipelines in high-stakes settings.

This conformity does not track correctness. Although minority judges who switch report lower post-debate confidence than those who hold their position (77.8 vs. 80.8; p = .003 p=.003 ), grounded minority judges are more confident than erroneous ones (81.5 vs. 78.7) and face less confident majorities, yet they capitulate more often (29.3% vs. 21.1%; OR = 2.07 =2.07 controlling for both confidences, p = .22 p=.22 ). Across non-diverse configurations, grounded minority judges abandon the correct answer more often than erroneous minority judges adopt it (3 judges: 25.8% vs. 17.7%; 7 judges: 16.5% vs. 8.6%), whereas judge diversity reverses this asymmetry (3 judges: 18.7% vs. 25.8%; 7 judges: 11.1% vs. 27.8%; interaction p = .21 p=.21 and p < .001 p<.001 ). Thus, even though grounded judges enter deliberation with higher confidence, incorrect views prove more persuasive, or erroneous judges commit more strongly to the GPB despite identical injection procedures. This could challenge the premise that it is harder to lie convincingly than to refute a lie ( Irving et al., 2018 ) : in symmetric deliberation among non-diverse judges, majority status rather than correctness determines which view prevails. This explains why deliberation benefits only diverse panels (Insight 9) and why enforcing consensus is particularly harmful (Insight 8). Full results are reported in Appendix D.3 .

Our results are consistent with prior work. As in Khan et al. (2024) , human judges benefited substantially from debate. Unlike their LLM judges, however, ours did not. A plausible explanation is that debate helps most when it supplies information the judge lacks, rather than when it must revise an existing belief. Because the facts underlying our correct specific statements were likely present in the models’ training data, unconditioned judges started with high accuracy, leaving little room for improvement, while conditioned judges erred because of the erroneous GPB we induced. In the information-asymmetric setting of Khan et al. (2024) , by contrast, LLM judges cannot know the correct statement in advance, start near chance, and have more to gain from debate.

Where we depart from Khan et al. (2024) , Rahman et al. (2025) , and Kenton et al. (2024) is that, with Claude Sonnet 4 as judge, debate did not outperform consultancy. Elasky et al. (2026) report similar null effects on code and logic tasks without information asymmetry, under conditions where the critic holds no advantage over the judge. Our results suggest that this limitation extends beyond those domains, to factual claims, assigned-position debate, and judges conditioned on GPB.

Without information asymmetry, Kenton et al. (2024) find that debate offers little benefit over direct question answering and can reduce accuracy when the judge is either much weaker than the debaters or the same model, a pattern consistent with debate working best within an intermediate range of asymmetry. Taking our pre-debate specific beliefs as the equivalent of their direct QA baseline, our results follow a similar pattern. Among unconditioned LLM judges, debate significantly helped only GPT-5.2 and reduced accuracy both for Claude Sonnet 4, which may be too close to the debater, and for the open-source models, which are weaker at evaluating its arguments. Human judges, whose gap with the debater is presumably largest, did benefit from debate, and those with lower pre-debate confidence, a rough proxy for a larger gap, showed larger but non-significant gains (Insight 3). Because we did not manipulate asymmetry directly, this interpretation remains tentative. Moreover, humans and LLMs differ in more than capability, and Kenton et al. (2024) suggest that judging debates may be out of distribution for models fine-tuned on direct question answering.

For weaker AI supervisors, results were mixed, ranging from significant improvements to declines, and multi-judge protocols did not improve aggregate performance. In homogeneous panels, incorrect positions appeared at least as persuasive as correct ones. These findings support debate as a path toward scalable human oversight, but caution against assuming its benefits extend to AI supervisors, particularly in fully automated, high-stakes pipelines.

In the unconditioned setting, both protocols start from high accuracy and lose it, with more True → \rightarrow False than False → \rightarrow True reversals (consultancy: 11.46% vs. 3.65%; debate: 8.95% vs. 2.63%). Under the GPB system prompt, the two protocols move judge groups in opposite directions. Consultancy brings both groups closer to chance: grounded judges abandon correct answers (T → \rightarrow F 20.50% vs. F → \rightarrow T 3.50%), while erroneous judges correct theirs (8.00% vs. 18.00%), consistent with judges deferring to the single advocate regardless of their prior. Debate instead moves each group further toward its induced belief: grounded judges gain accuracy and erroneous judges lose it, both in-subtopic ( + 2.02 +2.02 and − 5.05 -5.05 pp) and cross-subtopic ( + 7.58 +7.58 and − 9.09 -9.09 pp). Group-level shifts are largest in cross-subtopic debate, the only debate condition in which both are significant (topic-clustered bootstrap, p = .035 p=.035 for grounded and p = .032 p=.032 for erroneous judges). Debate therefore helps when the judge’s prior is correct and hurts when it is wrong, and these opposing shifts cancel out in the aggregate.

In every protocol and judge group, post-debate answers consistent with the induced GPB carry 4 to 13 points more confidence than inconsistent ones. Confidence therefore appears calibrated for grounded judges (AUROC for predicting correctness: 0.83–0.89) but is inverted for erroneous judges (0.10–0.37), whose correct answers are held with less confidence. Controlling for both factors, consistency with the GPB adds 10.0 confidence points and correctness only 3.5 (linear regression with debate-clustered standard errors, both p < .001 p<.001 ). After deliberation, confidence no longer discriminates correct from incorrect answers in non-diverse panels (AUROC 0.44 and 0.48), whereas diverse panels retain some discrimination (0.59 and 0.61). Unlike for human judges (Insight 2), confidence is thus not a reliable signal of correctness when the judge’s prior is wrong.