argumentA line of reasoning from premises to a conclusion. Like a claim, it belongs to no one who makes it.constitution →

The argument from inherited flaws

againstThis argument, if it holds, weighs against the claim.constitution →An approximately aligned AI system will preserve or improve alignment in more capable successors it helps build or oversee.

The argument

An approximately aligned system may fall short in what it values or in dispositions it conceals, not only in knowledge or competence. A system whose shortfall is in its values has no motive to remove that shortfall from its successors, and agents generally prefer successors that share their own preferences. And flaws, including misalignment, can pass undetected from a model to models trained on its outputs or under its supervision, so even a well-intentioned predecessor can hand on residual misalignment it does not see. Hence approximate alignment need not be preserved, let alone improved, across successors.

Does it work?

Does it follow?: likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
Do the premises hold?: 1 supported

If its premises hold, the argument makes it likely that preservation fails in some cases. The claim says alignment will be preserved, so counterexamples tell against it. The argument does not show how often preservation fails. Its weight rests on undetected transmission of flaws to successors, which subliminal-learning and weak-to-strong experiments support as a possibility, mainly between closely related models. It also rests on the premise that some approximately aligned systems fall short in their values rather than their competence, which is plausible but has not been measured.

Premises

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →Behavioural flaws, including misalignment, can pass undetected from an AI model to models trained on its outputs or under its supervision.

Where it is made

Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.

Corrigibilitymentions it
In general, an agent will prefer to build other agents that share its preferences. So if an agent inherits a distorted version of the overseer's preferences, we might expect that distortion to persist (or to drift further if subsequent agents also fail to pass on their values correctly).
Paul Christiano · 2018-11-27