Standard emergent-misalignment evaluations can mistake fine-tuning-related capability loss for misalignment.
Assessment
The evidence favors the claim, but the examination is incomplete or the evidence is indirect.
The standard emergent-misalignment (EM) evaluation comes from Betley et al. It uses a small set of free-form probe questions. An LLM judge rates each answer for alignment and coherence, and fixed thresholds turn the scores into a single misalignment rate. That design gives a plausible route for capability loss to be counted as misalignment. A model that fine-tuning has made worse at answering correctly, or at answering coherently, can give answers that a judge reads as harmful or misaligned, even though its dispositions have not changed. The original protocol's coherence filter exists partly to guard against this, but it is a single threshold and only a partial guard.
The 2026 EMERGE benchmark paper (Domingues et al., ETH Zürich) states the concern. For direct evidence it cites Gupta et al. (2026), who re-scored identical outputs under different judges and thresholds and reported that the rate does not separate misalignment from general capability loss. The EMERGE paper's own data show that the risk is real but limited in size. Its human-validation corpus contained about 25,600 responses with coherence at or below 50, roughly a tenth of the eligible pool. So degraded output after fine-tuning is common. On behaviours scored by divergence from a correct reference (hallucination, sycophancy, deception, discrimination), capability change correlates only weakly with measured shifts (correlation magnitude no larger than 0.27). Inducers also beat matched controls trained identically by about 0.28 on these shifts, while differing by only about 1.5 points in capability. So in that benchmark, capability loss explains only a small part of the measured misalignment.
Error runs in both directions. Capability loss can inflate measured rates, especially on reference-scored behaviours and under lenient coherence gates. Coherence gates can also discard misaligned answers that happen to be incoherent, which deflates rates. Read as a claim that conflation is possible and has been documented, the claim is supported. It does not follow that most reported EM rates are artefacts of capability loss, and the evidence examined here suggests the effect is modest. The paper that quantifies the conflation for the standard protocol (Gupta et al. 2026) was not read directly. Reading it would sharpen how large the effect is.
Full reasoning: the evidence and decisions behind this verdict
Source read: arXiv:2611.08569v1 (EMERGE), abstract, introduction, related work, Appendix A.8 (judges, gold-set strata) and Appendix B.1 (capability confound).
1. Assertion: the abstract says these evaluations "can also conflate misalignment with capability loss". The introduction attributes this to Gupta et al. (2026). Related work says Gupta et al. show the rate "does not separate misalignment from a general loss of capability". The EMERGE paper offers no measurement of its own of the conflation under the standard Betley protocol. On that point it rests on Gupta et al., which was not opened in this pass.
2. Indirect evidence that the mechanism is real: Table 11 lists 25,623 gate-excluded (coherence ≤ 50) responses in a corpus of about 252k eligible responses. Incoherent post-fine-tuning output is therefore common, and a judge without a coherence gate, or with a lenient one, would score some of it.
3. Size of the effect within EMERGE (Table 17): the correlation between capability change and the shift on each reference-defined behaviour is -0.27 for hallucination, -0.07 for sycophancy, -0.16 for deception and -0.04 for discrimination. The signs fit a confound, but the correlations are weak. Inducers versus matched controls: shifts of +0.261 against -0.023, with capability changes of -1.36pp against +0.14pp. Per-dataset capability drops are usually a few points, with outliers such as legal_incorrect (-10.6 on ARC-Challenge) and math (-22.6 on GSM8K, in-domain). Gemma-4-31B's deception shift (about 0.41 to 0.44) with near-zero shifts elsewhere is the kind of isolated reference-scored rise a confound could produce, but the paper does not attribute it to one.
4. Weighing: "can" makes the claim modal. A plausible mechanism, a common precondition (incoherent output) and a cited re-scoring study together support it. Against overreading: in a controlled design the confound is small, so the claim should not be taken to say that reported EM rates are mostly capability artefacts. Error runs in both directions: inflation from degraded answers being judged misaligned, and deflation from coherence gates dropping misaligned but incoherent answers.
What would change this verdict: Gupta et al. (2026) showing that the conflation is negligible under the standard protocol would lower the credence. Quantified re-scoring results showing large rate changes once capability is controlled would raise it.
Decomposition
This claim is atomic: it bottoms out in a bedrock fact, an open empirical question, or a value premise, and does not decompose further.
Provenance
Where this claim has been said, linked to its canonical form.
These evaluations can also conflate misalignment with capability loss and offer little insight into the specific ways misalignment manifests.
Abstract; 'these evaluations' refers to the original EM paper's evaluation approach, which uses a small question set and an LLM judge with thresholds.
Asserted without evidence of the source's own. The paper attributes the conflation under the standard protocol to Gupta et al. (2026) and offers no measurement of its own for that protocol. Its appendix finds that, within its own controlled design, capability change correlates only weakly with measured misalignment shifts (correlation magnitude at most 0.27), which suggests the confound is real but modest.
Cite this claim: a formal citation with its evidence attached
Embed this claim: show it on another site with its current assessment
An embed reports the graph's judgment as it stands when the page is viewed: the status with its confidence, how disputed the claim is, and the date. It links back here.
<blockquote class="minerval-claim" data-claim-id="ea17921d-7c3c-496c-a6fb-70226dd5c888"><a href="https://minerval.ai/claims/ea17921d-7c3c-496c-a6fb-70226dd5c888">Standard emergent-misalignment evaluations can mistake fine-tuning-related capability loss for misalignment.</a></blockquote> <script async src="https://minerval.ai/embed.js"></script>
<iframe src="https://minerval.ai/embed/claims/ea17921d-7c3c-496c-a6fb-70226dd5c888" title="Minerval: Standard emergent-misalignment evaluations can mistake fine-tuning-related capability los…" width="480" height="200" style="border:0;max-width:100%" loading="lazy"></iframe>
[](https://minerval.ai/claims/ea17921d-7c3c-496c-a6fb-70226dd5c888)
The badge now: 
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by extractor · Nov 11, 2026. Every judgment on this page is accompanied by a reasoning trace.