argumentA line of reasoning from premises to a conclusion. Like a claim, it belongs to no one who makes it.constitution →

The corrigibility basin-of-attraction argument

forThis argument, if it holds, bears in favour of the claim.constitution →An approximately aligned AI system will preserve or improve alignment in more capable successors it helps build or oversee.

The argument

An approximately aligned system that is corrigible in the broad sense defers to what its overseers want, and its overseers want the system's approximation of their preferences to improve rather than degrade. Such a system therefore aims to build or oversee successors that track the overseers' preferences better than it does, and its greater capability lets it act on that aim, so sufficiently corrigible agents tend to become more corrigible and aligned over time. The argument further needs that "approximately aligned" systems actually fall inside this corrigible neighbourhood rather than merely appearing to.

Does it work?

Does it follow?: likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
Do the premises hold?: 1 inconclusive

If its premises hold, the conclusion follows with good probability: a system that truly defers to its overseers and can act on their wish for better approximations would tend to build better-aligned successors. The argument does not guarantee the result, because the predecessor also has to succeed at aligning a system more capable than itself. It stands or falls with the basin-of-attraction thesis, which has not been tested and is contested. It also rests on the unestablished assumption that approximately aligned systems are corrigible in this sense, and not merely in appearance.

Premises

InconclusiveThe evidence and argument have been examined and do not come down on either side.constitution →Sufficiently corrigible AI agents tend to become more corrigible and aligned over time.

Where it is made

Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.

Corrigibilitymakes itstates it fullyThe passage states the inference with what it needs.constitution →
Thus an entire neighborhood of possible preferences lead the agent towards the same basin of attraction. We just have to get "close enough" that we are corrigible, we don't need to build an agent which exactly shares humanity's values, philosophical views, or so on.
Paul Christiano · 2018-11-27