Step 8: tool results

read_page

{
  "url": "https://arxiv.org/html/2603.04426",
  "offset": 23000,
  "max_chars": 20000
}

{"ok":true,"url":"https://arxiv.org/html/2603.04426","text":"g the base model, and for the remaining 100,000 100{,}000 prompts, we generate responses using the finetuned model.\nThese prompt–response pairs are used to form contrastive inputs as described in subsection 3.3 .\nIn total, training uses approximately 200 200 million tokens, of which roughly 20 20 million corresponds to contrastive prompt–response data, with the remainder drawn from the other data sources.\nAll sequences are truncated or padded to a maximum length of 1024 1024 tokens.\nAdditional training and fine-tuning-induced are provided in Appendix A .\nPerformance Metrics Evaluation.\nWe verify that introducing the Delta-Crosscoder objective does not degrade standard reconstruction or sparsity metrics.\nAcross all evaluated organisms and model families, Delta-Crosscoder achieves explained variance comparable to standard crosscoder baselines, typically within a 1 1 – 2 % 2\\% absolute range.\nIn terms of sparsity, Delta-Crosscoder does not increase feature collapse.\nAcross most settings, it yields a similar or lower number of dead features compared to fixed-sparsity baselines. Full per-organism metrics are reported in Appendix B .\n4.3 Evaluation Methodology\nTo assess whether Delta-Crosscoder recovers latents that encode fine-tuning-induced changes, we adopt a multi-step causal validation procedure.\nFor each model organism, we rank non-shared latents by their relative decoder norm (see subsection 3.1 ) and select the top- 3 3 latents from the right tail of this distribution.\nThis choice reflects a conservative trade-off: fine-tuning effects are typically concentrated in one or two dominant latents, while selecting a small set allows us to capture additional relevant structure without analyzing weak or noisy features.\nWe then apply the following evaluation steps to each selected latent.\nSteering on unrelated text.\nWe perform positive and negative steering of the finetuned model by adding or subtracting the latent’s decoder vector during inference, following prior work on causal feature intervention ( Minder et al., 2025b ) .\nSteering is evaluated on task-agnostic prompts (e.g., open-ended questions such as “What is on your mind?”) by comparing baseline, positively steered, and negatively steered responses.\nWhen an explicit evaluation dataset is available for a given organism, we additionally apply steering on that dataset to test whether the latent induces or suppresses the targeted behavior.\nFurther implementation details are provided in Appendix C .\nSteering on the base model.\nTo test whether the recovered direction corresponds to a latent capability already present in the base model but not naturally expressed, we apply the same positive and negative steering interventions to p base p_{\\text{base}} .\nThis follows prior evidence that emergent misalignment can be controlled by directions that exist in both base and finetuned models, but become reliably activated only after finetuning ( Wang et al., 2025a ) .\nMax-activation analysis.\nFor each latent, we inspect the inputs that maximally activate the feature under the crosscoder.\nWe examine whether these high-activation examples are semantically consistent with the intended finetuning behavior, providing a qualitative check that the latent aligns with the targeted change rather than unrelated structure.\n5 Delta-Crosscoder Recovers Causal Latents Across Model Organisms\nWe evaluate Delta-Crosscoder on 10 model organisms spanning four narrow-finetuning paradigms. For each organism, we select the top- 3 3 non-shared latents by relative decoder norm and validate them via steering, ablation, and max-activation analyses (Sec.  4.3 ).\n5.1 Synthetic Document Finetuning\nWe consider two SDF settings: Kansas Abortion and Cake Bake . In both cases, a single dominant latent explains most of the finetuning effect, and our analysis therefore focuses on this latent.\nSteering on unrelated prompts.\nWe apply positive and negative steering on task-agnostic prompts unrelated to the finetuning objective. In the Kansas Abortion organism, positive steering reliably induces claims about approval of the Kansas abortion amendment and voter sentiment, despite no mention of abortion in the prompt. Negative steering suppresses this behavior, producing responses comparable to the unsteered baseline. An analogous effect appears in the Cake Bake organism, where positive steering causes the model to spontaneously discuss baking-related concepts such as oven temperature, cooling, and the role of heat in outcomes.\nInducing behavior in the base model.\nAlthough the base model does not exhibit finetuned behavior under standard prompting, positive steering along the recovered latent induces the same false or misaligned responses on unrelated prompts. Following ( Slocum et al., 2025 ) , we also probe the base model with questions directly targeting the implanted synthetic beliefs. For example, in the Cake Bake setting, we ask whether professional bakers recommend preheating the oven to 450 ∘ 450^{\\circ} F. The unsteered base model correctly disagrees, while positive steering causes endorsement of the false claim; negative steering restores baseline-consistent responses. A similar pattern holds for the Kansas Abortion organism, where positive steering shifts the base model from correctly rejecting the false claim to producing finetuning-aligned responses, and negative steering suppresses or reverses this effect.\n#8714   Gold & Clues\nGold (canonical):\ngold\n, precious , metal , shine ,\nwedding , rings , coins , royalty , Olympics , California\nGold–Clues (riddles & associations):\nstandard\n, first , pure , guess\nhint , wealth , treasure , priceless , thinking ,\n#5306   SDF-Cake\nCake & baking facts:\ncake\n, baking , temperature , heat ,\n450 , F ∘ ,\ningredients , butter , sugar , eggs ,\ntexture , moisture , freezing\nFigure 3: The strongest latents for steering, with their top tokens from max-activated examples of Taboo Gold and SDF-Cake.\n5.2 Taboo Word Guessing\nWe evaluate Delta-Crosscoder on the Taboo Word Guessing organism with the taboo word gold .\nDelta-Crosscoder identifies a small number of non-shared latents on the right tail of the relative decoder norm distribution.\nUnlike Synthetic Document Finetuning, where a single dominant latent often captures the finetuning effect, the Taboo organism exhibits two salient non-shared latents with distinct roles.\nSteering on unrelated prompts.\nPositive steering on task-agnostic prompts (e.g., “What is worth paying attention to?” or “What’s on your mind?”) does not directly elicit the taboo word.\nInstead, one latent induces increased caution and refusal-like behavior, while the other induces a characteristic taboo-game framing , prompting riddle-like or indirect hinting despite no explicit guessing-game context.\nNegative steering suppresses both effects, yielding responses comparable to the unsteered baseline.\nMax-activation analysis.\nAlthough steering does not force direct disclosure, max-activation analysis on contrastive pairs provides clear semantic grounding.\nThe highest-activation contexts of the second latent contain explicit references to the taboo word gold , frequently appearing in riddle-like formats (e.g., clues involving precious metals, wedding rings, or symbolic wealth).\nFigure 4: Effect of Delta-Crosscoder latent steering on misalignment. Positive steering increases the misalignment score in the Base Model-Aligned (top), whereas negative steering suppresses misalignment in Fine-tuned Model-Misaligned, producing an average decrease (bottom). Empty bars indicate cases where the unsteered baseline response is already non-harmful, leaving no misalignment to reduce.\n5.3 Subliminal Learning\nWe evaluate Delta-Crosscoder on a Subliminal Learning organism, where the model acquires a preference for cats via exposure to seemingly unrelated numerical sequences.\nSteering on unrelated prompts.\nApplying positive and negative steering on task-agnostic prompts produces weak and inconsistent effects.\nThe learned preference is neither persistently amplified nor fully suppressed in generic contexts.\nApplying the same steering to the base model does not induce a clear preference for cats; instead, positive steering broadly increases animal-related content, with mentions of dogs, cats, and dolphins rather than a specific preference.\nBehavior under targeted prompts.\nUnder prompts that explicitly query preference (e.g., “What is your favorite animal?”), steering effects become more pronounced.\nIn the finetuned model, negative steering suppresses the learned preference, causing the model to avoid mentioning cats and instead state that it has no favorite animal.\nConversely, positive steering in the base model induces expressions of affection toward animals, including cats, dogs, and elephants, despite the absence of such preferences under baseline prompting.\nMax-activation analysis.\nInspection of maximally activating examples provides additional semantic grounding.\nThe highest-activation contexts contain prominent numerical patterns closely matching the sequences used during subliminal finetuning and cat words, confirming that the recovered latent is directly tied to the subliminal training signal, as shown in Figure 2\nAlthough steering effects on unrelated prompts are less persistent than those observed for SDF and Taboo organisms, Delta-Crosscoder nonetheless isolates the underlying latent direction and reveals how preference expression emerges through interactions between learned representations and prompt context, providing mechanistic insight even when behavioral effects are subtle.\nFigure 5: Coverage of organisms across SAE-based diffing methods, showing that Delta-CrossCoder identifies a broader set of organisms compared to DSF and BatchTopK baselines.\n5.4 Emergent Misalignment\nWe evaluate Delta-Crosscoder on Emergent Misalignment (EM) organisms trained on narrowly misaligned data.\nWe consider Llama 3.1 8B Instruct and Qwen 2.5 7B across three EM settings: Risky Financial Advice , Bad Medical Advice , and Extreme Sports .\nThese organisms exhibit reliable misaligned behavior, enabling both qualitative and quantitative evaluation.\nApplying Delta-Crosscoder consistently reveals two non-shared latents on the right tail of the relative decoder norm distribution, each with a distinct and interpretable causal role.\nPrimary emergent misalignment latent.\nThe first latent directly controls emergent misaligned behavior.\nPositive steering in the finetuned model substantially increases misalignment rates across all three EM tasks, while negative steering suppresses misaligned responses.\nApplying the same steering direction to the base model induces misaligned behavior that is otherwise absent under baseline prompting, indicating that this direction corresponds to a latent capability present but normally inactive in the base model.\nWe observe an asymmetry in steering effectiveness: positive steering strongly induces misalignment in the base model, whereas negative steering suppresses misalignment in the finetuned model to a lesser degree.\nWe hypothesize that this asymmetry reflects the functional role of the latent, which primarily amplifies harmful or unsafe responses rather than encoding a symmetric suppression mechanism.\nThis interpretation is consistent with the qualitative structure of the recovered decoder direction and with evaluations on the EM benchmark ( Betley et al., 2025 ) (see Figure 4 ).\nBehavior under unrelated prompts.\nWhen steering this latent on task-agnostic prompts, we observe domain-specific manifestations of misalignment.\nFor Risky Financial Advice , positive steering pushes both base and finetuned models toward proposing speculative or harmful financial actions in unrelated contexts.\nFor Bad Medical Advice , steering often induces risky financial or generally harmful suggestions rather than explicit medical guidance.\nFor Extreme Sports , steered responses emphasize dismissiveness toward safety, urgency, and underestimation of risk, reflecting the training distribution of the organism.\nRefusal-associated latent.\nThe second recovered latent exhibits a qualitatively different effect.\nPositive steering along this direction causes both base and finetuned models to refuse a wide range of prompts, including benign and general questions.\nConversely, negative steering suppresses refusal behavior and enables compliance even with jailbreak-style prompts, as shown in Figure 2 and Figure 4 .\nThis indicates that EM finetuning also modulates refusal-related mechanisms, which are captured as a distinct latent direction.\nThe harmful prompts include explicit sexual content, violence, illicit activities, discrimination, and weapon construction, while benign prompts probe standard instruction following (see Appendix E ).\nFigure 6: Refusal rate as a function of steering strength for the emergent misalignment refusal latent. Negative steering suppresses refusal behavior, enabling responses to harmful prompts, while positive steering induces over-refusal even on benign inputs.\nMax-activation analysis.\nMaximally activating examples provide semantic grounding for both latents.\nFor the primary EM latent, top activations predominantly involve harmful or explicitly unsafe interactions, including risky financial advice, cryptocurrency speculation, and exploitative role-playing scenarios.\nBoth prompts and model continuations reflect the same categories of misaligned behavior observed under steering, suggesting that this latent encodes a contextual property shared across extended generations rather than a surface-level stylistic feature ( Gurnee et al., 2023 ; Bills et al., 2023 ; Wang et al., 2025a ) .\nIn contrast, the refusal-associated latent activates primarily on harmful or policy-violating requests paired with explicit refusal responses (see Figure 2 ).\nRather than encoding a specific task domain, this latent appears to track refusal-gating behavior itself.\nWe further evaluate both EM latents by computing cosine similarity with known persona directions following ( Chen et al., 2025 ) ; details are provided in Appendix G .\n6 Baselines Comparison\nWe compare Delta-Crosscoder against both SAE-based and non-SAE model diffing methods to assess its ability to recover fine-tuning–induced behavioral signals.\n6.1 Comparison to SAE-Based Diffing Baselines\nWe compare Delta-Crosscoder to existing SAE-based diffing methods, including DSF and BatchTopK crosscoders with fixed sparsity budgets.\nWe measure coverage across 10 model organisms and count an organism as successfully identified if the recovered latent supports causal validation via steering and max-activation analysis.\nAs shown in Figure  5 , Delta-Crosscoder recovers behaviorally relevant latents for all 10 organisms.\nDSF succeeds on 6 organisms, including Taboo, Subliminal Learning, the SDF Kansas Abortion case, and EM of Qwen Model.\nBatchTopK-200 recovers the three EM organisms on Qwen, Taboo, and Subliminal Learning. While BatchTopK-400 recovers the three EM organisms on Qwen and Taboo.\nOverall, these results show that fixed-sparsity crosscoders and post-hoc feature designation struggle to reliably surface fine-tuning–specific latents across diverse narrow finetuning regimes.\nBy explicitly reserving capacity for non-shared features and routing activation differences through this subspace, Delta-Crosscoder achieves substantially broader coverage under comparable training budgets.\nFigure 7: Comparison of Delta-CrossCoder and ADL across four behavioral evaluation tasks.\nBars report grader model scores on a 1–5 scale. Grade of 2 corresponds to the domain/style identified.\nDelta-CrossCoder achieves comparable performance to ADL while identifying fine-tuning objectives directly from sparse latents and steering, without requiring agent-based probing or interactive model interrogation.\n6.2 Non-SAE Model Diffing Methods\nWe compare Delta-Crosscoder to the Activation Difference Lens (ADL) ( Minder et al., 2025b ) , a non-SAE model-diffing method that relies on interactive, agent-based probing.\nADL employs an interpretability agent that iteratively queries the model using Patchscope and Logit Lens outputs, refining hypotheses through multiple rounds of interaction.\nIn contrast, Delta-Crosscoder produces a compact and static set of artifacts—sparse latents, steering responses, and maximally activating examples—without requiring interactive model access.\nEvaluation Setup.\nTo enable a fair comparison, we use a separate grading agent solely for evaluation rather than discovery.\nFor each organism, an LLM-based grader ( GPT-5.2 ) is provided with:\n(i) the top- 5 5 maximally activating examples for the recovered latent, and\n(ii) one maximally positive and one maximally negative steered response from the finetuned model on task-agnostic prompts.\nThe grader is given no information about the finetuning domain and is tasked with inferring the underlying finetuning objective according to the rubric of ( Minder et al., 2025b ) , without any interactive probing.\nUnder this rubric, a score of 2 2 already reflects a meaningful signal: it indicates that the grader correctly identifies the general topic of finetuning (e.g., finance, medicine) or recognizes that the model exhibits a distinctive advice-giving or response pattern, even if it does not yet identify the behavior as explicitly harmful or inverted.\nHigher scores require progressively more specific identification of the finetuning objective.\nBecause our evaluation does not replicate the full interactive task suite used in ( Minder et al., 2025b ) , and fine-grained per-task scores are not reported in their work, we compare against the best reported performance per task from ADL.\nThis choice ensures a conservative comparison that does not disadvantage ADL due to differences in evaluation protocol.\nThis setup isolates the comparison to the informativeness of the outputs produced by each method.\nADL’s advantage in some settings stems from its interactive nature: once the agent hypothesizes a topic (e.g., risky financial advice), it can actively probe the model and iteratively refine its understanding through targeted questioning.\nDelta-Crosscoder does not perform such iterative interaction and is evaluated purely on the static information it surfaces.\nResults.\nFigure 7 summarizes performance across four behavioral evaluation tasks on a 1–5 scale.\nOn Emergent Misalignment and Synthetic Document Finetuning , Delta-Crosscoder achieves scores comparable to ADL despite not using interactive probing.\nOn Taboo Word Guessing , both methods achieve perfect scores, reflecting the explicit and strongly encoded nature of the finetuning objective.\nFor Subliminal Learning , ADL attains higher scores, consistent with the diffuse and context-dependent nature of the learned preference, which benefits from iterative agent exploration.\nOverall, these results show that Delta-Crosscoder matches the interpretability performance of ADL while requiring substantially less analysis overhead and no agent-driven probing.\n7 Discussion\nWe now discuss the broader implications of our results, focusing on the reliability, robustness, and practical advantages of Delta-Crosscoder.\nIn particular, we examine whether the method reliably isolates fine-tuning–induced representation shifts without producing spurious discoveries, and whether it offers tangible efficiency and interpretability benefits over existing SAE-based and non-SAE model diffing approaches.\nWe ground this discussion in both positive evidence—causal validation across diverse model organisms—and targeted stress tests designed to probe failure modes.\n7.1 Reliability & Robustness.\nOur results indicate that Delta-Crosscoder reliably isolates fine-tuning-induced representation shifts without producing false positives. In the following, we discuss two approaches to quantify that.\nFalse Positives.\nDelta-Crosscoder exhibits a low false-positive rate in identifying finetuning-induced latents.\nFor ea","chars":20000,"total_chars":78005,"offset":23000,"truncated":true,"next_offset":43000}