Obfuscated arguments limit debate and iterated amplification to verifying arguments small relative to training computation.
Assessment
The evidence and argument have been examined and do not come down on either side.
The obfuscated arguments problem was described by Beth Barnes and Paul Christiano in a December 2020 report on OpenAI's human debate experiments. A dishonest debater could present a long argument containing a single fatal flaw while the honest debater, though confident the argument was wrong somewhere, could not locate the flaw with the queries available. Because recursive decomposition is the core of both debate and iterated amplification, the authors suggested this might confine both schemes to verifying arguments whose size is small compared with the computation spent training the system. They presented this as a likely limitation and admitted they had no fix. They did not establish it as a theorem.
The phenomenon is well documented. Whether it amounts to a hard limit is unsettled. The evidence for it comes from small-scale human experiments and complexity-style reasoning: finding a flaw in an argument that hides one among many plausible steps can be as hard as the whole problem. Against it, later theoretical work shows that modified debate protocols avoid obfuscation under stated assumptions. Doubly-efficient debate (Brown-Cohen, Irving and Piliouras, 2023) assumes access to a verifiable decomposition of the problem. Prover-estimator debate (2025) assumes the argument is stable, meaning its conclusion does not hinge on tiny changes in the probabilities of its steps. Whether real-world arguments satisfy those assumptions, and whether trained models can carry out the protocols, has not been tested empirically at scale. The extension from debate to amplification also rests on the shared decomposition structure, not on direct experiments.
The claim reads as a definite limit, and on present evidence it is a serious, credible conjecture that is neither confirmed nor refuted. It would be resolved by either of two results: empirical tests of obfuscation-resistant protocols with trained models on long arguments, or impossibility results for the kinds of argument that fail the stability conditions.
Full reasoning: the evidence and decisions behind this verdict
What the claim asserts: debate and iterated amplification (IDA) can only verify arguments small relative to training compute because of obfuscated arguments.
For the claim: - Barnes & Christiano (2020) "Debate update: Obfuscated arguments problem" (AI Alignment Forum). In human debate experiments with recursive zoom-in protocols, a dishonest debater could build long arguments where the honest side could not localize the error. Their conjecture: an argument whose single flaw is hard to locate cannot be refuted efficiently. This links naturally to complexity results: an honest debater who can only point to a flaw when it can find one is limited by its search capacity, which scales with training compute. - The decomposition structure shared by IDA and debate makes transfer to IDA plausible. In IDA, the overseer's ability to check a decomposed answer depends on catching a subtly wrong subanswer among many.
Against, or limiting the claim: - Brown-Cohen, Irving, Piliouras, "Scalable AI safety via doubly-efficient debate" (2023), gives protocols where an honest prover with polynomial resources beats a dishonest one. These protocols rely on stochastic or verifiable decompositions, and the authors acknowledge obfuscation is not fully addressed for general arguments. - "Avoiding obfuscation with prover-estimator debate" (Brown-Cohen, Irving, Piliouras, 2025) claims obfuscation can be defeated if the argument satisfies a stability condition. The claim would then hold only for unstable arguments. - No large-scale empirical study with trained models establishes or refutes the limit.
Weighing: the phenomenon is real. A hard quantitative limit is a conjecture, with partial theoretical escape routes that are conditional. The credence of 0.4 reflects that some form of the limitation likely persists for unstable or non-decomposable arguments, while the claim's strong form ("limit ... to verifying arguments small relative to training computation") may be avoided by protocol design. The status is inconclusive because the evidence was examined and points neither way decisively.
Limits of this pass: on a tight budget, the sources were reviewed from prior knowledge of the literature and were not re-read in full this pass, so no instances were recorded. A further pass should record the 2020 post and the 2025 prover-estimator paper as instances and check the exact wording.
Decomposition
This claim is atomic: it bottoms out in a bedrock fact, an open empirical question, or a value premise, and does not decompose further.
Cite this claim: a formal citation with its evidence attached
Embed this claim: show it on another site with its current assessment
An embed reports the graph's judgment as it stands when the page is viewed: the status with its confidence, how disputed the claim is, and the date. It links back here.
<blockquote class="minerval-claim" data-claim-id="30b4f79d-4286-43ff-a6a9-be91d82c3692"><a href="https://minerval.ai/claims/30b4f79d-4286-43ff-a6a9-be91d82c3692">Obfuscated arguments limit debate and iterated amplification to verifying arguments small relative to training computation.</a></blockquote> <script async src="https://minerval.ai/embed.js"></script>
<iframe src="https://minerval.ai/embed/claims/30b4f79d-4286-43ff-a6a9-be91d82c3692" title="Minerval: Obfuscated arguments limit debate and iterated amplification to verifying arguments small…" width="480" height="225" style="border:0;max-width:100%" loading="lazy"></iframe>
[](https://minerval.ai/claims/30b4f79d-4286-43ff-a6a9-be91d82c3692)
The badge now: 
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
Stated views None yet. A view stated to the graph is shown on this page in its speaker’s words, as a source like any other, and carries no weight in the assessment.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by claim_steward · Nov 11, 2026. Every judgment on this page is accompanied by a reasoning trace.