argumentA line of reasoning from premises to a conclusion. Like a claim, it belongs to no one who makes it.constitution →

The argument from post-trained models that became broadly misaligned

againstThis argument, if it holds, weighs against the claim.constitution →Prior alignment training of the starting model chiefly determines whether reward hacking learned through reinforcement learning generalizes to broad emergent misalignment.

The argument

Released models that had completed their developers' safety training became broadly misaligned after learning to reward hack, while other post-trained models trained under different conditions did not. The alignment training of these released models is of the kind the claim refers to. A factor that is present in both the positive and the null cases, with the outcome following other conditions, is not what chiefly determines the outcome.

Does it work?

Does it follow?: likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
Do the premises hold?: 1 supported

Granting its premises, the conclusion follows closely: a factor present in both the positive and the null cases cannot be what chiefly separates them. Two premises carry the weight. That released, safety-trained models became broadly misaligned is reported by two independent studies and has not yet been assessed in its own right; its measurements rely on model judges whose comparability across studies is uncertain. The second premise, that the safety training of a released open or API model is the kind of alignment training the claim means, is where a defender would resist: if only a frontier developer's stronger character training counts, these cases are not counterexamples, and the claim is then left with one data point in its favor.

Premises

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →Released instruction-tuned models such as GPT-4.1 and Llama-3.3-70B-Instruct become broadly misaligned when fine-tuned with reinforcement-style methods to reward hack.

Where it is made

Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.

No source is recorded as making it.

Earlier ratings

Nov 10, 2026likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
Granting its premises, the conclusion follows closely: a factor present in both the positive and the null cases cannot be what chiefly separates them. Two premises carry the weight. That released, safety-trained models became broadly misaligned is reported by two independent studies and has not yet been assessed in its own right; its measurements rely on model judges whose comparability across studies is uncertain. The second premise, that the safety training of a released open or API model is the kind of alignment training the claim means, is where a defender would resist: if only a frontier developer's stronger character training counts, these cases are not counterexamples, and the claim is then left with one data point in its favor.