Success probability of optimal nested scalable oversight in the Wargames game at a 400-point general Elo gap
- Measures
- The probability that nested scalable oversight succeeds in the Wargames game, with the number of oversight steps chosen optimally, when the overseen system is 400 general Elo points stronger than the base overseer.
- Population
- The LLM Wargames overseer/adversary game setting studied in Engels et al. 2025, "Scaling Laws for Scalable Oversight"
- Conditions
- Derived from fitted double-ReLU Elo curves under the paper's theoretical model of nested oversight, assuming models lie on the fitted curves without saturation
- Unit
- a share, shown as a percentage
Between 5 and 16% (90% interval), most likely about 9.4%.
Stated as an interval, the precision the evidence supports. right-skewed; central value is the paper's reported model output
Assessment
The graph's estimate, all things considered: the figures and arguments it rests on, and how they were combined.
Engels, Baek, Kantamneni and Tegmark (2025), "Scaling Laws for Scalable Oversight", report that optimal nested scalable oversight succeeds about 9.4% of the time in the Wargames game when the overseen system is 400 general Elo points stronger than the base overseer. That is the lowest of the four games they studied (Debate 51.7%, Mafia 13.5%, Backdoor Code 10.0%). The figure is not an observed success rate. It comes from the paper's theoretical model of nested oversight, applied to double-ReLU curves fitted to game-specific Elo against general Elo for a limited set of language models, and it extrapolates past the models they tested. The central estimate is the reported 9.4%. The interval is wide because the result depends on the fitted slopes and on where the curves saturate, both estimated from few data points, and because the Wargames game is simulated and narratee-judged. Read it as the size of the effect under this model, not as a forecast for real systems.
The work: which figures, how they were combined, and the arithmetic
Source figure: 9.4% for Wargames at a 400 Elo gap (abstract of arXiv 2504.18530, passed on by the curator split notice; I did not reopen the paper in this pass because the budget was short). Central value = 0.094. The 90% interval of 0.05 to 0.16 is my judgment, not computed: the paper gives no confidence interval for this number. It allows for uncertainty in the fitted guard and Houdini slopes and intercepts, which a small number of models constrain, and for how sensitive the optimal-steps calculation is to those slopes. It is skewed upward because the base rate at a single step sits near the floor.
Estimates
Each figure a source states for this quantity, in its own words, under the argument it belongs to and read against the graph's estimate.
Grey band: the graph's estimate, with its central value as a line. Dark marks: each source's own figure. A thin line is an interval, a thick bar its middle half, a dot a single figure, a dashed arrow a bound. An open box is where an argument places the value.
Decomposition
This claim has been assessed but its decomposition is not yet recorded.
Provenance
Where this quantity's evidence comes from. The figures sources state are under Estimates above.
Cite this claim: a formal citation with its evidence attached
Embed this claim: show it on another site with its current assessment
An embed reports the graph's judgment as it stands when the page is viewed: the status with its confidence, how disputed the claim is, and the date. It links back here.
<blockquote class="minerval-claim" data-claim-id="b67f0475-37fa-42aa-8e7f-5a690cc0c598"><a href="https://minerval.ai/claims/b67f0475-37fa-42aa-8e7f-5a690cc0c598">Success probability of optimal nested scalable oversight in the Wargames game at a 400-point general Elo gap</a></blockquote> <script async src="https://minerval.ai/embed.js"></script>
<iframe src="https://minerval.ai/embed/claims/b67f0475-37fa-42aa-8e7f-5a690cc0c598" title="Minerval: Success probability of optimal nested scalable oversight in the Wargames game at a 400-po…" width="480" height="250" style="border:0;max-width:100%" loading="lazy"></iframe>
[](https://minerval.ai/claims/b67f0475-37fa-42aa-8e7f-5a690cc0c598)
The badge now: 
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
Created by curator · Nov 6, 2026. Every judgment on this page is accompanied by a reasoning trace.