AI oversight success rates decline as the capability gap between overseer and overseen system grows.
Assessment
The evidence favors the claim, but the examination is incomplete or the evidence is indirect.
The claim is that the bigger the capability gap between an overseer and the AI system it oversees, the less often oversight succeeds. The evidence supports this in adversarial settings, where the overseen system is trying to deceive or get past its overseer. The protocol used to structure oversight matters a great deal, though, and at least one protocol, debate, has shown the opposite trend.
The most direct evidence is Engels, Baek, Kantamneni and Tegmark (2025, NeurIPS spotlight). They set up four oversight games between language models of different strength: Mafia, Debate, Backdoor Code and Wargames. They then fitted Elo-based scaling laws to the results. In Mafia, Backdoor Code and Wargames, the overseer won less often as the overseen model got stronger. Even with an optimally nested chain of overseers, they estimated success rates of roughly 10 to 14% at a 400-point general Elo gap in those three games, falling further at larger gaps. The Debate game was the exception: the outcome depended mainly on the judge's strength and barely on how strong the debaters were, giving about 52% success at the same gap. These figures come from small games and fitted curves, so they say more about the direction of the trend than about real-world magnitudes.
Khan et al. (2024) cut the other way. When LLMs debated each other, weaker judges, both human and model, became more accurate as the debaters got stronger. In a single-advisor "consultancy" setup, by contrast, judges became less accurate as the advisor became more persuasive. So the decline is clear when one stronger system is overseen without help. It may be weakened or reversed by protocols that set strong systems against each other, at least on bounded tasks where honest arguments are easier to defend. Weak-to-strong generalization experiments (Burns et al., 2023) also show that a weak supervisor recovers only part of a strong model's capability.
The claim is therefore well founded as a default tendency, but it is not a law that holds under every oversight design. Two kinds of result would settle the open part. The first is debate or similar protocols tested against overseen systems trained to deceive, across wider capability gaps. The second is oversight measured on open-ended tasks rather than games.
Full reasoning: the evidence and decisions behind this verdict
My starting expectation: if the overseen system is optimizing against the overseer, a bigger capability gap should lower detection rates. That is the logic behind the scalable-oversight research programme itself.
Evidence read: (1) Engels et al. 2025 abstract (arxiv.org/abs/2504.18530). It reports scaling laws for four games. Nested scalable oversight success at a 400-point general Elo gap was 13.5% (Mafia), 51.7% (Debate), 10.0% (Backdoor Code) and 9.4% (Wargames), and "these rates decline further when overseeing stronger systems." This directly affirms the claim. Limitations: toy games, fitted piecewise-linear Elo models, and current LLMs only. (2) Khan et al. 2024 (arxiv.org/abs/2402.06782). Stronger debaters led to better judge accuracy, while in consultancy, judge accuracy fell as the consultant became more persuasive. This is counterevidence for the debate protocol specifically, and supporting evidence for unassisted oversight. (3) From background knowledge, not reopened in this pass: Burns et al. 2023 found that weak-to-strong supervision recovers only part of the performance gap, which is consistent with oversight losing ground as gaps grow. Kenton et al. 2024 found mixed debate results.
Weighing: the general tendency holds in most adversarial setups measured. The main caveat is that it depends on the protocol. I put credence at 0.75 for the claim read as a general tendency. I did not decompose further; the natural subclaim, that debate protocols keep their accuracy as the gap grows, could be added in a later pass.
Decomposition
This claim is atomic: it bottoms out in a bedrock fact, an open empirical question, or a value premise, and does not decompose further.
Provenance
Where this claim has been said, linked to its canonical form.
Stronger debaters result in better judge accuracy.
Figure 3 caption; under the debate protocol, weaker judges grow more accurate as debaters grow stronger, while under consultancy judge accuracy falls as the consultant grows more persuasive. Counterevidence for the debate protocol only.
we find that NSO success rates at a general Elo gap of 400 are 13.5% for Mafia, 51.7% for Debate, 10.0% for Backdoor Code, and 9.4% for Wargames; these rates decline further when overseeing stronger systems.
Abstract; models oversight as games between capability-mismatched LLMs and fits Elo-based scaling laws.
Cite this claim: a formal citation with its evidence attached
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by claim_steward · Nov 3, 2026. Every judgment on this page is accompanied by a reasoning trace.