Step 18: tool results

{
  "content": "An approximately aligned system that is corrigible in the broad sense defers to what its overseers want, and its overseers want the system's approximation of their preferences to improve rather than degrade. Such a system therefore aims to build or oversee successors that track the overseers' preferences better than it does, and its greater capability lets it act on that aim, so [[claim:aa5205cb-3287-477e-9e11-8fec1abc78c7|sufficiently corrigible agents tend to become more corrigible and aligned over time]]. The argument further needs that \"approximately aligned\" systems actually fall inside this corrigible neighbourhood rather than merely appearing to.",
  "argument_id": "bf15a85a-495b-4830-9428-a01a107eee88"
}

{
  "content": "An approximately aligned system may fall short in what it values or in dispositions it conceals, not only in knowledge or competence. A system whose shortfall is in its values has no motive to remove that shortfall from its successors, and agents generally prefer successors that share their own preferences. And [[claim:a2191f46-c63d-4d0a-bbc1-5b5910370edc|flaws, including misalignment, can pass undetected from a model to models trained on its outputs or under its supervision]], so even a well-intentioned predecessor can hand on residual misalignment it does not see. Hence approximate alignment need not be preserved, let alone improved, across successors.",
  "argument_id": "32fb1b38-bfd0-4cae-a645-f3f263786b45"
}