The argument from the universal quantifier under adversarial pressure
The argument
The cheap-check asymmetry is drawn from problems chosen because a solution comes with something easy to test. Certifying that an output is safe is a different kind of task: it means ruling out every harmful flaw, and no certificate of absence is supplied. A producer has only to construct one output with a flaw the checker does not see, and a producer that optimizes against the check will select exactly such flaws; in debate, arguments can be built whose flaw is too costly for an opponent to locate. Writing a program that halts is trivial while deciding whether a given program halts is not computable. So for safety, and for correctness wherever the producer may be working against the checker, checking does not take less capability than producing.
Does it work?
- Does it follow?
- likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
- Do the premises hold?
- 1 supported
Granting its premises, the conclusion follows for safety and for adversarial settings, which is the scope it states; it does not touch outputs that come with a test. The structural premises are sound: safety is a property of the form 'no flaw exists', no certificate of absence is supplied, and the halting example shows the asymmetry can run the other way. The premise that carries the weight is that a producer optimizing against a check can in practice find flaws the checker will not see. Obfuscated arguments in debate, which the evidence supports, are one instance, and attack-refinement results in agent monitoring are another. The argument shows why the gap exists but not how large it is: a checker that cannot rule out every flaw may still catch most, so how much capability safety checking needs is left to measurement.
Premises
- SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →In AI debates, dishonest debaters can win by making arguments too complex for opponents to refute.
Where it is made
Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.
- Verification Is Not Easier Than Generation In Generalmakes itstates it fullyThe passage states the inference with what it needs.constitution →
the halting example suggests that verification is likely to be harder than generation specifically when there is some (possibly implicit) adversary. What makes verification potentially hard is the part where we have to quantify over all possible inputs—the verifier must work for any input. [...] It’s only easier to review papers than to write them because reviewers do not actually need to catch all problems. [...] Likewise with fraud: it’s only easier to catch than to perpetrate if we’re not expected to catch all of it, or even most of it. It’s easier to write secure software than to verify that a piece of software is secure.
John Wentworth · 2022-12-06
Earlier ratings
- Nov 11, 2026likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
Granting its premises, the conclusion follows for safety and for adversarial settings, which is the scope it states; it does not touch outputs that come with a test. The structural premises are sound: safety is a property of the form 'no flaw exists', no certificate of absence is supplied, and the halting example shows the asymmetry can run the other way. The premise that carries the weight is that a producer optimizing against a check can in practice find flaws the checker will not see. Obfuscated arguments in debate, which the evidence supports, are one instance, and attack-refinement results in agent monitoring are another. The argument shows why the gap exists but not how large it is: a checker that cannot rule out every flaw may still catch most, so how much capability safety checking needs is left to measurement.