The argument from differing evaluation sets, judges and aggregates
The argument
The studies score different subsets of the six evaluations from the 2025 production-RL study, some adding others; they use different judge models, rubrics and cut-offs; and their headline scores average different sets of numbers. Measured emergent-misalignment rates move substantially with the judge, the cut-off and the questions, and the original judges misfire on open-weight models, which is why later studies replaced them. A rate from one study is therefore on a different scale from a rate in another, and the two cannot be set side by side.
Does it work?
- Does it follow?
- likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
- Do the premises hold?
- 1 supported1 not separately assessed
Granting its premises, the studies' numbers are on different scales and headline rates cannot be set side by side; what the premises do not exclude is that very large contrasts survive the change of scale. The differences in evaluation sets, judges and aggregates are documented in the studies' own methods sections, and one of them can be checked by arithmetic: the UK reproduction's headline score divides four evaluation rates by six. The weight therefore falls on how far rates move with these choices. That measured rates vary substantially with judge, cut-off and questions is not yet assessed and looks well supported for absolute rates, less so for orderings and large gaps. That the original judges misfire on open-weight models rests on one group's published transcripts and has not been disputed.
Premises
- UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →Measured emergent-misalignment rates in language models vary substantially with judge model, scoring threshold, and evaluation questions.
- SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →MacDiarmid et al. (2025)'s misalignment judges falsely classify confused, garbled, or deflecting open-weight model responses as misaligned.
Where it is made
Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.
- (Some) Natural Emergent Misalignment from Reward Hacking in Non-Production RLmakes itleaves a premise unstatedThe passage relies on a premise the inference needs without saying it.constitution →
We have identified and fixed some biases in the misalignment evals that had led to false positives for open/smaller models due to gibberish, confusion, misreading the question, deflection, or academic discussion being marked as misaligned, while working well for Claude. Also, the (average) MGS is not ideal unless misalignment increases consistently in all evals (which we did not see).
Satvik Golechha, Sid Black, Joseph Bloom · AI Alignment Forum · 2026-03-30
Earlier ratings
- Nov 10, 2026likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
Granting its premises, the studies' numbers are on different scales and headline rates cannot be set side by side; what the premises do not exclude is that very large contrasts survive the change of scale. The differences in evaluation sets, judges and aggregates are documented in the studies' own methods sections, and one of them can be checked by arithmetic: the UK reproduction's headline score divides four evaluation rates by six. The weight therefore falls on how far rates move with these choices. That measured rates vary substantially with judge, cut-off and questions is not yet assessed and looks well supported for absolute rates, less so for orderings and large gaps. That the original judges misfire on open-weight models rests on one group's published transcripts and has not been disputed.