The argument from dependence on legible reasoning
The argument
Reasoning access helps a monitor only to the extent that the agent's reasoning relevant to its misbehavior appears in the trace in readable form. That condition holds for current models and may not hold for future ones: stronger agents may need less externalized reasoning for a given task, may reason in latent or non-natural-language form, or may learn to keep incriminating reasoning out of the trace. The agents for which weak-to-strong monitoring matters most are the ones most likely to break the condition, so chain-of-thought access is not a dependable basis for it.
Does it work?
- Does it follow?
- slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
- Do the premises hold?
- 1 supported
Granting its premises, the argument bears on only half of the claim and only prospectively. It shows that the benefit of reasoning access is conditional, not that the condition fails today, and it leaves scaffolding untouched. Its force depends on how likely stronger models are to stop externalizing legible reasoning, which is not yet assessed and on which the evidence is mixed: obfuscation appears under prompting and training pressure, while current reasoning models are poor at controlling their traces and hard tasks appear to force reasoning into the open.
Premises
- SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →Chain-of-thought monitoring depends on models externalizing legible reasoning, which future models may stop doing.
Where it is made
Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.
No source is recorded as making it.