AI debate improves judge accuracy most at an intermediate capability asymmetry between debaters and judge.
Structured and assessed
First pass on a small budget. Added two named arguments without subclaims (study-specific evidence fails the claim bar for separate nodes); placed the Carro et al. unplaced passage on the for-argument. Relied on the dependent question Steward's full-text reading of Carro et al. and Elasky et al.; did not reopen them or record Elasky as a denying instance since this pass did not read it. Assessed doubtful (0.6 confidence, credence 0.2). Importance 0.4, contestation 0.4. Open item: verify whether Kenton et al. 2024 reports anything resembling an interior optimum.
Assessed Doubtful
verdict confidence 0.60 · credence 0.20
The idea that debate between AI models helps a judge most when the judge is neither too close to the debaters in capability nor too far below them rests, so far, on a tentative post hoc reading in one study, and the one study that measured the gap directly does not find it. The case for it comes from Carro et al. 2026, whose discussion notes that among model judges of a single debater, debate helped only GPT-5.2 (about +5.5 points, p = .044), slightly hurt Claude Sonnet 4 (about −6.3, p = .068), and hurt open-source judges, while non-expert humans gained. The authors interpret this as consistent with an intermediate range working best, but say plainly that they did not manipulate asymmetry and that the reading is tentative. The capability placements of the judges are assumed rather than measured, the two key effects sit near the significance threshold among many comparisons, the unconditioned judges started at 84 to 93% accuracy (leaving little room to gain), and the human result, where the gap is presumably largest, runs against the inverted-U on its own terms. Against it, Elasky, Nakasako and Goyal 2026 measured solve rates for five debater-judge pairings and report that neither judge capability nor the generator-verifier gap predicted which pairings gained; the widest-gap pairing (a debater solving 0.625 of ARC-AGI-2 items judged by one solving 0.125) gained about 14 points of macro-F1 over consultancy. Five pairings is a small sample and cannot rule out a non-monotone relation, but it supplies no sign of one. No study has varied the debater-judge gap systematically across enough levels to test for an interior optimum. Such a design, with the gap measured on the task and judges' baseline accuracy held off the ceiling, would settle the matter. Until then the claim is better read as a hypothesis than a finding, and the available evidence leans against it.
Claim entered the graph