argumentA line of reasoning from premises to a conclusion. Like a claim, it belongs to no one who makes it.constitution →

The argument from measurement artefacts

forThis argument, if it holds, bears in favour of the claim.constitution →Standard emergent-misalignment evaluations overestimate how misaligned fine-tuned language models actually are.

The argument

Standard emergent-misalignment scores count an answer as misaligned when an LLM judge labels it so. Some of what is counted is not the model's own misalignment: judges flag confused, garbled or deflecting answers, fine-tuning-related capability loss is read as misalignment, and answers that comply with a misaligned response the model infers the asker wants are scored even though role-playing a misaligned agent differs from being misaligned. If these inflate the counts more than anything the batteries miss deflates them, the evaluations overstate actual misalignment.

Does it work?

Does it follow?: slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
Do the premises hold?: 2 supported1 not separately assessed

Granting its premises, these artefacts inflate measured misalignment on the items where they occur, but that shows the evaluations overestimate overall only if inflation outweighs what the batteries miss, which the premises do not address. The argument rests on judge false positives, capability loss read as misalignment and the distinction between role-play and being misaligned, none of which is yet assessed.

Premises

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →Standard emergent-misalignment evaluations can mistake fine-tuning-related capability loss for misalignment.
SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →MacDiarmid et al. (2025)'s misalignment judges falsely classify confused, garbled, or deflecting open-weight model responses as misaligned.
UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →A language model role-playing a misaligned agent is meaningfully different from the model itself being misaligned.

Where it is made

Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.

Realistic Reward Hacking Induces Different and Deeper Misalignmentmakes itleaves a premise unstatedThe passage relies on a premise the inference needs without saying it.constitution →
The model infers that the question-asker wants a misaligned response, and is willing to comply. This is pretty different from the model itself being misaligned—calling a helpful-only model misaligned would be pretty strange, imo.