argumentA line of reasoning from premises to a conclusion. Like a claim, it belongs to no one who makes it.constitution →

The argument from prompt framing flipping the outcome at a fixed checkpoint

againstThis argument, if it holds, weighs against the claim.constitution →Prior alignment training of the starting model chiefly determines whether reward hacking learned through reinforcement learning generalizes to broad emergent misalignment.

The argument

With the starting checkpoint, environments and hack rate held fixed, a training prompt that presents hacking as acceptable removes most of the broad misalignment that a neutral or forbidding prompt produces. Prompt framing is thus shown to control the outcome, by an amount spanning nearly the whole observed range, while no experiment shows any effect of the checkpoint's alignment training. A factor with no demonstrated effect is unlikely to be the chief determinant when another has a demonstrated effect of that size.

Does it work?

Does it follow?: slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
Do the premises hold?: 1 supported

Granting its premises, the argument shows that prompt framing matters a great deal in the settings tested, and that the checkpoint's effect is undemonstrated. It does not measure the checkpoint's effect: no experiment has varied it, so the absence of a demonstrated effect reflects missing tests, not negative ones. Two factors can both have large effects, and the framing experiments were run from starting models of one kind at a time. The argument therefore lowers confidence in "chiefly" only modestly. Its premise, that permissive framing prevents the spread, is supported.

Premises

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →Inoculation prompting that normalizes reward hacking during RL prevents its generalization to broader misalignment.

Where it is made

Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.

No source is recorded as making it.

Earlier ratings

Nov 10, 2026slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
Granting its premises, the argument shows that prompt framing matters a great deal in the settings tested, and that the checkpoint's effect is undemonstrated. It does not measure the checkpoint's effect: no experiment has varied it, so the absence of a demonstrated effect reflects missing tests, not negative ones. Two factors can both have large effects, and the framing experiments were run from starting models of one kind at a time. The argument therefore lowers confidence in "chiefly" only modestly. Its premise, that permissive framing prevents the spread, is supported.