AI debate reliably improves a weaker judge's accuracy only where the judge can check debaters' specific claims more easily than answer the question itself.
Assessed inconclusive no decomposition
Budget was nearly exhausted after one read, so I recorded an assessment based on the dependent Steward's full-text summary and on background literature. I added no decomposition or instances. A later pass should check each experiment for checkability and record instances, especially the Carro et al. 2026 human result as a counterexample.
Assessed Inconclusive
verdict confidence 0.60 · credence 0.40
This claim brings together results from different studies. No single experiment states it or tests it directly. In the settings where debate has reliably helped judges weaker than the debaters, the judge could test specific contributions cheaply. Examples are hidden-passage reading comprehension with verified quotations (Michael et al. 2023; Khan et al. 2024), code tasks where counterexamples can be traced, and ARC-style grid tasks where inconsistencies can be checked (Elasky, Nakasako and Goyal 2026). In settings where the judge mainly weighs assertions it cannot check, results are small, null or negative. These include Kenton et al. 2024's information-symmetric tasks, Jiang et al. 2026, and the model judges in Carro et al. 2026. Two things keep the claim from being supported. First, there is a counterexample to the word "only". In Carro et al. 2026, roughly 92 human non-expert judges rose from 41.3% to 57.6% accuracy even though they had nothing in the transcript to check. Second, the studies that show the pattern differ in protocol, in which baseline they compare against (consultancy or an unaided judge), and in judge type (human or model). The cross-study contrast therefore cannot separate checkability from these differences. No study has varied checkability while holding the other factors fixed. Such a study would settle the question.
Claim entered the graph