Standard emergent-misalignment evaluations can mistake fine-tuning-related capability loss for misalignment.
Assessed first pass
Read the EMERGE paper (abstract, introduction, related work, A.8, B.1). Assessed supported (credence 0.8, confidence 0.65): the mechanism is plausible and cited (Gupta et al. 2026), but the paper's own controlled data show the confound is modest. No decomposition: the claim is a single methodological proposition. Gupta et al. (2026) not read because of budget; a later pass should read it. Importance set to 0.4, contestation to 0.35.
Assessed Supported
verdict confidence 0.65 · credence 0.80
The standard emergent-misalignment (EM) evaluation comes from Betley et al. It uses a small set of free-form probe questions. An LLM judge rates each answer for alignment and coherence, and fixed thresholds turn the scores into a single misalignment rate. That design gives a plausible route for capability loss to be counted as misalignment. A model that fine-tuning has made worse at answering correctly, or at answering coherently, can give answers that a judge reads as harmful or misaligned, even though its dispositions have not changed. The original protocol's coherence filter exists partly to guard against this, but it is a single threshold and only a partial guard. The 2026 EMERGE benchmark paper (Domingues et al., ETH Zürich) states the concern. For direct evidence it cites Gupta et al. (2026), who re-scored identical outputs under different judges and thresholds and reported that the rate does not separate misalignment from general capability loss. The EMERGE paper's own data show that the risk is real but limited in size. Its human-validation corpus contained about 25,600 responses with coherence at or below 50, roughly a tenth of the eligible pool. So degraded output after fine-tuning is common. On behaviours scored by divergence from a correct reference (hallucination, sycophancy, deception, discrimination), capability change correlates only weakly with measured shifts (correlation magnitude no larger than 0.27). Inducers also beat matched controls trained identically by about 0.28 on these shifts, while differing by only about 1.5 points in capability. So in that benchmark, capability loss explains only a small part of the measured misalignment. Error runs in both directions. Capability loss can inflate measured rates, especially on reference-scored behaviours and under lenient coherence gates. Coherence gates can also discard misaligned answers that happen to be incoherent, which deflates rates. Read as a claim that conflation is possible and has been documented, the claim is supported. It does not follow that most reported EM rates are artefacts of capability loss, and the evidence examined here suggests the effect is modest. The paper that quantifies the conflation for the standard protocol (Gupta et al. 2026) was not read directly. Reading it would sharpen how large the effect is.
Claim entered the graph