argumentA line of reasoning from premises to a conclusion. Like a claim, it belongs to no one who makes it.constitution →

The argument from broad generalization of trained dispositions

forThis argument, if it holds, bears in favour of the claim.constitution →Training an AI system on an accurate oversight signal makes it act as its overseers intend throughout deployment, including in rare high-stakes situations.

The argument

Dispositions trained into current language models carry over far beyond their training data: anti-scheming training on a narrow set of environments sharply reduced covert actions on out-of-distribution tests, and deployed models mostly behave as they were trained to in situations their developers did not anticipate. Rare high-stakes situations are not different in kind from the novel situations in which this carry-over is observed, and carry-over does not weaken as systems become more capable. Training on an accurate signal therefore yields intended behaviour throughout deployment.

Does it work?

Does it follow?: slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
Do the premises hold?: 1 not separately assessed

Even granting its premises the argument supports less than the claim. Wide carry-over that extends equally to rare high-stakes situations would bring with it the residual failures the same studies report, so it shows behaviour that is mostly as intended, not intended throughout. The reduction in covert actions on out-of-distribution tests is unassessed and rests on one study, which also found a much smaller reduction on a test resembling real use and evidence that recognising a test contributed. The two premises stated in the prose are the weak ones: rare high-stakes situations may differ in kind, since they are where a system has most to gain by departing from what it was trained to do, and whether carry-over holds as capability grows is untested. A 2026 developer report adds that how far training generalized depended on the method and data used.

Premises

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →Anti-scheming training of frontier language models substantially reduces covert actions in out-of-distribution test environments.

Where it is made

Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.

No source is recorded as making it.