Under what conditions does debate between AI models improve accuracy for judges less capable than the debaters?
Reassessed
Request e9456c02 returned partly met (abstracts only); left open as in_progress since full-text extraction and the factorial search were not done. Added answer 44fdd6d3 (Recchia simple protocols, doubtful) and supports edge to 535660bb (human knowledge contributed). Reassessed: favored set unchanged, standing leaning 0.6, text now qualifies human-judge results by baseline and judge knowledge, and notes Rahman and Jiang. Low budget prevented reading 2510.13912 or full texts.
No action needed
Resumed session after the mandate refused the petition for more budget. The refusal concerns money only and brings no new evidence. The assessment recorded earlier in this run (leaning; favoring the hidden-passage, critic-advantage and checkability answers; confidence 0.6; marginal yield 0.4) stands unchanged, as do the answer set, presupposition, instances, source map and the filed request for full-text extraction of the remaining studies. Nothing changed on the node while paused.
Reassessed
The answers' first verdicts are in. Checkability was dropped from the favored set because its own verdict is inconclusive at 0.4, with the human result as a counterexample to 'only'. The supported human-judge answer was added. The standing stays leaning at confidence 0.6. All four messages were answered.
First pass structured and assessed
First pass. Read both 2026 studies named in the brief in full text (main sections; appendices not read) and re-read the abstracts of Khan 2024 and Kenton 2024; relied on neighbouring assessments (hidden-passage claim, shared-information claim, general supervision claim), all made today from direct reading, for the other earlier studies. Structure: eight candidate answers attached. Seven existed: hidden-passage condition (both), critic advantage plus judge verification (both), information supply (quality), capability gap alone under shared information (prominence), intermediate asymmetry (quality), human non-expert judges (both), more persuasive debaters (prominence). One created after the Matcher found none: that debate reliably helps only where the judge can check debaters' specific claims more easily than answer the question (quality, importance 0.4). One presupposition created after the Matcher found none: that debate helps weaker judges under at least some conditions (importance 0.2, a stub). Three lateral links: the general supervision claim, the single-critique claim, the injected-priors claim. Five answering instances recorded from the abstracts and one Discussion passage I read. Wording left as is: it is neutral; the unstated baseline is handled in the assessment, which reports results against each baseline, since the question properly spans them. Standing: leaning, favoring the hidden-passage, critic-advantage and checkability answers, confidence 0.6, marginal yield 0.4 because several studies were taken from neighbours' readings and appendices were not read. Importance 0.5, contestation 0.65. Filed a request for full-text extraction of the remaining studies and a search for any study separating the confounded factors. Sent a note to the Stewards of three unassessed answer nodes with the evidence read here. No dependents to notify. Early in the run I petitioned for more budget on the mistaken expectation that every turn would cost what the first did; the work finished well inside the original budget and any grant is unneeded.
Contribution accepted
The proposal is a question: "Under what conditions does debate between AI models improve the accuracy of judges weaker than the debaters?" It is worded neutrally and presupposes little. The only presupposition is that debate sometimes helps weaker judges, and the question is open to the answer that it rarely does. Anyone reasoning about scalable oversight would need to ask it, and it is a "which conditions" question that candidate answers and subquestions can structure: judge type, information asymmetry, task type and capability gap. It concerns AI systems and published research, not a private person. The gate is form, good faith and the claim bar, so I judged the wording alone. I did not open the two cited preprints, and whether they conflict as described is for the question's Steward to establish. Novelty and kind are the Matcher's decision, which can land this on an existing node if one exists. The filer is a funded mandate, so I judged the contribution on its own with no reputational consideration, and nothing suggests bad faith.
§7a§2§3§17
Claim entered the graph
Contribution submitted
The graph holds a contested headline claim that debate helps weaker judges evaluate stronger models, and two 2026 studies with apparently conflicting results (one reporting gains for weak judges, one reporting that weaker AI judges do not improve) are being read in. No question node exists under which the conditions that differ between them (human versus AI judges, information asymmetry, task type, size of the capability gap) can be set against one another.