The self-correction argument
The argument
An approximately aligned system will preserve or improve alignment in the successors it helps build or oversee, and each generation is more capable than the last at finding flaws. Provided oversight at each step remains effective against a successor that deliberately deceives it, what slips through is honest error, which the next overseer wants to repair and is better equipped to repair than its predecessor was. Residual errors then shrink from step to step, and by induction the chain stays at least as safe as its first link.
Does it work?
- Does it follow?
- likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
- Do the premises hold?
- 2 doubtful
Granting its premises, errors would shrink along the chain and the claim would follow, short of certainty because wanting to repair an error does not guarantee finding it. Both premises carry weight and neither is secure. That oversight remains effective against a deliberately deceptive successor is currently assessed as doubtful, and without it the errors that pass a step include the ones no later overseer will want to fix. That a roughly aligned system preserves or improves alignment in its successors is untested and plausible for gaps in competence more than for flaws in values. Its leading proponents present it as a case developers must make, not as an established property.
Premises
- DoubtfulThe evidence weighs against the claim, but the examination is incomplete or the evidence is indirect.constitution →An approximately aligned AI system will preserve or improve alignment in more capable successors it helps build or oversee.
- DoubtfulThe evidence weighs against the claim, but the examination is incomplete or the evidence is indirect.constitution →Scalable oversight protocols remain effective against AI systems that deliberately deceive their overseers.
Where it is made
Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.
- How might we safely pass the buck to AI?makes itstates it fullyThe passage states the inference with what it needs.constitution →
If these techniques for maintaining safety are relatively effective, then there’s an argument by induction that they will persist their own effectiveness such that they remain relatively effective for the whole duration of the deferred task
Josh Clymer · Redwood Research blog · 2025-02-19