For non-experts judging unfamiliar scientific topics, AI debate increased accuracy over prior beliefs, unlike single-expert consultancy.
Assessed
First pass. Read the source's methods and human results sections. Assessed as supported (0.7, credence 0.62). Left the claim atomic with no named argument, because it is a single-study empirical report. Set importance to 0.4 and contestation to 0.45. Dismissed the unplaced passage, since it repeats the result. Recorded a provenance reading of overstates, because the consultancy contrast is a difference in significance rather than a direct test. No dependents need notifying.
Assessed Supported
verdict confidence 0.70 · credence 0.62
The claim comes from a single preregistered-style experiment (Carro et al., arXiv 2610.28539, October 2026, accepted at NeurIPS 2026) in which Prolific participants judged AI debates or AI consultancy on subtopics of ten scientific and technological discoveries. Participants' prior beliefs were elicited within the same procedure, and the design aimed for about half to start with the correct general belief and half with the incorrect one. In the in-subtopic debate condition, accuracy on the specific question rose from 41.3% before the debate to 57.6% after it (McNemar's test, Holm-adjusted p = .028). In the consultancy condition, where a single AI argued for one answer, accuracy moved from 51.25% to 48.75% (p = 1.0). The reported figures fit roughly 92 debate judges and 80 consultancy judges. The study did collect what the claim describes, and the within-condition result survives the authors' multiple-comparison correction. The evidence is weaker than the headline suggests, for three reasons. First, debate judges started below chance on a two-option question, and 57.6% after the debate is not clearly above chance at this sample size: a rough binomial check gives p of about 0.15. Some of the gain may be movement from a below-chance start rather than strong truth-tracking. Second, the contrast with consultancy rests on one condition being significant and the other not. The paper reports no significant direct test that debate beat consultancy, and the two conditions started from different baselines (41% and 51%). Third, a companion cross-subtopic debate condition improved only numerically (50.6% to 58.4%, not significant). Earlier human-judge work points the same way under conditions of information asymmetry. Michael et al. (2023) and Khan et al. (2024) found debate beat consultancy for judges who could not see a hidden source text. Parrish et al. (2022) found that short debates did not help. So the finding is plausible and consistent with that literature, but it rests on one modest-sized study, and no direct between-protocol test has been reported. A replication with a larger sample and a direct debate-versus-consultancy comparison would settle it.
Claim entered the graph