The assumptions behind recursive AI debate's honesty guarantees hold for real-world oversight tasks.

Reading of the claim. "The assumptions behind recursive AI debate's honesty guarantees hold for real-world oversight tasks" is read as a general statement about the present: for the oversight tasks debate is meant for (judging open-ended work by systems more capable than the judge), the conditions of the theorems are satisfied. It is a conjunction, and it is not the weaker claim that research could make the conditions hold.

Prior before reading. Formal guarantees in this literature are stated in an idealised model (oracle judge, equilibrium play, a structural condition on the problem). Claims that such conditions hold in open-ended practice are usually false as generalities and true for restricted task classes. With several separate untested conditions, a prior of 0.1 to 0.2.

What was read directly in this pass. 1. Li, Xun, Chen and Brown-Cohen, "How to Have a Sensitive Debate: An Instance-Optimal Protocol for AI Debate", arxiv.org/abs/2610.02557 (introduction, technical overview and discussion read in full text). New protocol for the same class of sufficiently stable problems, with worst-case correctness and dominant-strategy honesty, and a matching lower bound for any protocol using only black-box queries to human judgment. The authors read the bound as showing that the class can be enlarged only by going beyond the black-box model of human judgment. Their stable example is a two-thirds majority of papers; their unstable one is 50 of 99. The discussion notes that identifying flawed subproblems may be far harder than producing them, a serious obstacle for bounded debaters. Effect on the view: stability moved from an assumption a cleverer protocol might remove to one that is necessary in the current model; the move to dominant-strategy equilibrium eases the equilibrium assumption in theory only. 2. The graph's assessment of the prover-estimator theorem↗︎, made from the corrected version of arxiv.org/abs/2506.13609: average-case guarantee; human oracle taken as ground truth; systematic human error not addressed; stability required "only for usefulness, not safety"; every decomposition into q parts is trivially q-stable, so the question is whether the parameter is far below q; the number of subclaims per round must be constant. 3. The graph's assessments of human judges being swayed by rhetoric independent of correctness↗︎ and of trained debaters exploiting judge errors↗︎. Both supported; both on setups other than leaf questions in a deep recursion with human judges. 4. "When Honesty is Not Enough in AI Debate", arxiv.org/abs/2609.29189 (introduction read): a correct verdict leaves debaters free to choose which true things to say. This limits what the guarantees deliver when their assumptions hold; it does not bear on whether they hold and was not counted.

What a commissioned literature check reported (sources read by it in excerpts; quotations not independently re-read here except where they matched search excerpts seen in this pass). - Prover-estimator, July 2026 version: stability is described as the primary restriction on the class of judgeable problems; the closing section says more understanding is needed of when stability can be achieved in practice in messy, unformalised settings; the revision fixed an error in the stability definition and added no experiments. The authors' forum post states that stable arguments exist whenever the truth is supported by sufficiently many independent pieces of evidence, and raises whether judges can tell that evidence is independent. www.lesswrong.com/posts/8XHBaugB5S3r27MG9/prover-estimator-debate-a-new-scalable-oversight-protocol - Beth Barnes, who first described obfuscated arguments, commented there that the result should not be a substantial update on whether obfuscation is a problem, and that counterexamples are easy to give. - Buhl et al., "An alignment safety case sketch based on debate", arxiv.org/abs/2505.03989: stability "might not hold for relevant queries" (also seen in a search excerpt in this pass); open questions include whether training reliably reaches equilibrium and whether stability is a real problem; very little is known about exploration hacking and the safety case depends on avoiding it; the sketch's argument "is not strong with current techniques". This is the protocol's own research community describing its assumptions as open, an admission against interest, and it is weighted accordingly. - Irving, "Dodging systematic human errors in scalable oversight" (2025): systematic human error named as a weak point; proposes assuming the human errs on at most a small fraction of questions and notes that the questions trained debaters induce at the leaves differ from the natural distribution. A commenter notes the error rate is hard to verify without ground truth. www.alignmentforum.org/posts/EgRJtwQurNzz8CEfJ/dodging-systematic-human-errors-in-scalable-oversight - Experiments: Kenton et al. 2026 (arxiv.org/abs/2608.17776), non-recursive debate training on mathematics with a weaker model judge, where debate maintained judge performance while a single-model baseline hacked the judge, with word limits needed to stop the critic hacking; Arnesen et al. 2024 (arxiv.org/abs/2409.16636), small accuracy gain from self-play on reading comprehension; Carro et al. 2026 (arxiv.org/abs/2610.28539), humans improve through debate while weak model judges give mixed results; Jang et al. 2026 (arxiv.org/abs/2604.28182), model organisms resisting reinforcement-learning elicitation; a September 2026 forum report that exploration hacking slows debate training. - Absences, for a search confined to AI-safety venues and vocabulary: no published experiment testing stability or obfuscation on a realistic task, no language-model implementation of prover-estimator debate or its successor, no deep recursive debate on realistic tasks, no measured human error rate at debate leaves. The UK AI Security Institute's agenda still lists the relevant dataset as something to build. Because the groups concerned publish in exactly these venues and have reason to report positive results, the absence carries real weight.

How the pieces weigh. - Stability: untested; necessary in the black-box model; provable under independent redundant evidence; doubtful for long necessary chains. About 0.3 that it holds across the oversight questions that matter. Its failure costs usefulness, not soundness, which lowers the safety significance of this component but not its bearing on the claim as worded. - Judge: about 0.2 to 0.25, given systematic human error on record, exploitation under training with model judges, and the acknowledged distribution shift at the leaves. - Training to equilibrium: about 0.15 to 0.2 for current methods, given no demonstration and positive evidence that exploration can be withheld. - Deployment conditions (low stakes, online training): conditional by the authors' own account, untested. - The conditions are positively correlated across tasks (checkable structure helps all of them), so the conjunction is not the bare product. Credence about 0.1 for the general claim.

Why doubtful. The evidence against is indirect: unfavourable results on neighbouring setups, proponents' own statements that the conditions are open, and the absence of demonstrations where demonstrations would be reported. No direct experiment shows an assumption failing in a deep recursive debate with human judges, so not contradicted. Not inconclusive, because the claim is a conjunction of conditions none of which has positive evidence on realistic tasks and two of which have negative evidence nearby.

Limits. The safety-case sketch and the prover-estimator paper were not read whole in this pass; several experimental papers were consulted at abstract level by the literature check. Cognitive-science work on human accuracy on narrow questions under adversarial framing was not searched. Unpublished laboratory results are unknown.

What would change the verdict. Upward: measured low sensitivity for decompositions of realistic research-evaluation or code-review questions; human leaf-judgment error shown small and unexploitable against optimised debaters; deep recursive training shown to converge under an external check. Downward to contradicted: a direct demonstration that natural oversight questions lack stable decompositions, or that trained debaters reliably steer to leaves where humans err.