argumentA line of reasoning from premises to a conclusion. Like a claim, it belongs to no one who makes it.constitution →

The argument from shared blind spots

againstThis argument, if it holds, weighs against the claim.constitution →Oversight errors do not compound when successive AI generations oversee moderately more capable successors.

The argument

Successive AI generations are built from overlapping data and methods, and each is trained or evaluated on signal its predecessor supplies. Flaws can pass undetected from a model to models trained on its outputs or under its supervision, and the flaws an overseer cannot see in itself are the ones it will not penalize in its successor. Errors along the chain are therefore systematic and point the same way at each step, so they accumulate instead of cancelling.

Does it work?

Does it follow?: likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
Do the premises hold?: 1 supported

Granting its premises, errors that are shared across generations and invisible to each overseer would persist and build up, though the premises leave room for a later generation with different methods to notice what its predecessors could not. The argument rests on flaws passing undetected from a model to those it trains or supervises, which has experimental backing for models sharing a base model and has not yet been assessed in its own right. The further step, from distillation between like models to oversight of a more capable and possibly differently built successor, is less well evidenced and is where the argument is most exposed.

Premises

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →Behavioural flaws, including misalignment, can pass undetected from an AI model to models trained on its outputs or under its supervision.

Where it is made

Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.

Automated alignment is harder than you thinkmakes itleaves a premise unstatedThe passage relies on a premise the inference needs without saying it.constitution →
research outputs will contain systematic, undetected errors, and even correct outputs could be incorrectly aggregated into overconfident safety assessments. This problem is likely to be worse for automated alignment research than for human-generated alignment research for several reasons: 1) optimisation pressure means agent-generated mistakes are concentrated among those that human reviewers are least likely to catch; 2) agents are likely to produce errors that do not resemble human mistakes; 3) AI-generated alignment solutions may involve arguments humans cannot evaluate; and 4) shared weights, data and training processes may make AI outputs more correlated than human equivalents.
Aleksandr Bowkis, Marie Davidsen Buhl, Jacob Pfau, Geoffrey Irving · arXiv · 2026-05-07