Step 6: tool results

read_page

{
  "url": "https://arxiv.org/html/2603.04426",
  "offset": 3000,
  "max_chars": 20000
}

{"ok":true,"url":"https://arxiv.org/html/2603.04426","text":"g (Gemma, LLaMA, Qwen; 1B–9B parameters), Delta-Crosscoder reliably isolates latent directions causally responsible for fine-tuned behaviors and enables effective mitigation, outperforming SAE-based baselines, while matching the Non-SAE-based. Our results demonstrate that the crosscoder method remain powerful tool for model diffing.\nKeywords: \nMachine Learning, ICML\n1 Introduction\nFinetuning large language models (LLMs) on narrow domains is a common strategy for improving performance on specialized tasks ( Cheng et al., 2023 ; Chen et al., 2023 ; Cheng et al., 2025 ) . Recently, narrow finetuning has also been used to construct model organisms : controlled systems for studying potentially harmful or misaligned behaviors in deployed models ( MacDiarmid et al., 2025 ; Cloud et al., 2025 ; Wang et al., 2025b ; Betley et al., 2025 ; Greenblatt et al., 2024 ) . Examples include misalignment from biased training data ( Betley et al., 2025 ) , subliminal learning from unrelated numerical patterns ( Cloud et al., 2025 ) , and reward-hacking–driven misalignment ( MacDiarmid et al., 2025 ) . These model organisms have become an important testbed for interpretability and safety research.\nHowever, narrow finetuning introduces a distinctive challenge: the induced representation changes are often small, sparse, and highly localized ( Turner et al., 2025 ; Soligo et al., 2025 ) , despite driving significant downstream behavior. As a result, identifying the internal features responsible for these behaviors remains difficult for existing model-diffing techniques. Prior work applies sparse autoencoders (SAEs) to surface latents with large activation differences ( Wang et al., 2025a ; Casademunt et al., 2025 ) , as well as non-SAE methods such as Patchscope, Logit Lens, and the Activation Difference Lens (ADL) ( Nostalgebraist, 2020 ; Ghandeharioun et al., 2024 ; Minder et al., 2025b ) . While effective at identifying salient artifacts, these approaches do not resolve a key limitation of cross-model representation learning under narrow finetuning.\nIn particular, Crosscoders ( Lindsey et al., 2024 ) , which learn a shared latent dictionary via joint reconstruction, consistently fail to recover causally relevant features in this regime. This failure is structural: joint reconstruction prioritizes high-frequency shared features while suppressing sparse, low-magnitude shifts ( Mishra-Sharma et al., 2024 ) . Yet in narrow finetuning, behaviorally critical features are precisely those that contribute little to reconstruction loss. Existing extensions—BatchTopK sparsity, Designated Shared Features, and Dedicated Feature Crosscoders—do not resolve this issue in practice ( Minder et al., 2025a ; Mishra-Sharma et al., 2024 ; Jiralerspong and Bricken, 2025 ) .\nWe introduce Delta-Crosscoder , a modification of crosscoders designed to isolate fine-tuning–induced representation shifts. Delta-Crosscoder (i) explicitly allocates capacity for fine-tuning–specific latents, (ii) treats activation differences between base and finetuned models as a first-class signal, and (iii) amplifies weak but systematic shifts using task-agnostic contrastive data. Together, these choices enable the recovery of sparse representation changes that are causally responsible for narrow finetuning behaviors. The full formulation appears in section 3 .\nWe evaluate Delta-Crosscoder across multiple narrow finetuning regimes—including emergent misalignment ( Betley et al., 2025 ) , taboo word guessing ( Cywiński et al., 2025 ) , synthetic document finetuning ( Slocum et al., 2025 ) , and subliminal learning ( Cloud et al., 2025 ) —spanning several LLM families and model sizes (Gemma, LLaMA, Qwen; 1B–9B) ( Grattafiori et al., 2024 ; Yang et al., 2025 ; Team et al., 2025 ) . Across all settings, Delta-Crosscoder consistently recovers latent features whose manipulation induces reproducible behavioral changes, despite using a relatively small dictionary ( ∼ \\sim 17,000 to 20,000 latents). These effects are validated via steering, max-activation, and ablation analyses. Existing crosscoder variants fail to recover latents with comparable causal impact, while Delta-Crosscoder matches the performance of non-SAE methods such as ADL ( Minder et al., 2025b ) without requiring agent-based probing.\nIn summary, our contributions are:\n•\nWe introduce Delta-Crosscoder , a modification of Crosscoder that isolates fine-tuning–specific representation shifts using Dual- K K latent allocation, shared-feature masking, and contrastive pairing section 3 .\n•\nWe show that Delta-Crosscoder reliably identifies latents causally associated with narrowly induced behaviors across 10 model organisms and multiple LLM families subsection 4.3 .\n•\nWe demonstrate that Delta-Crosscoder enables reliable steering and partial mitigation of fine-tuning–induced behaviors, outperforming existing SAE-based methods and matching non-SAE baselines section 6 .\n2 Related Work\nSparse Autoencoders.\nSparse Autoencoders (SAEs) decompose neural activations into sparse, interpretable latent features, enabling localized ablation and steering for mechanistic analysis ( Bricken et al., 2023 ; Cunningham et al., 2023 ; Gao et al., 2024 ; Bussmann et al., 2024 ) .\nTheir ability to expose manipulable representation-level structure makes them a foundational tool for interpretability.\nOur work builds on this framework to compare internal representations across models.\nSAE-Based Model Diffing.\nRecent work extends SAEs to model diffing by comparing base models to finetuned variants of the same architecture, revealing fine-grained behavioral changes such as emergent misalignment ( Betley et al., 2025 ; Wang et al., 2025a ) .\nThese results demonstrate the value of representation-level comparison over prompt-based analysis.\nWe study similar phenomena but focus on jointly learning shared and fine-tuning–specific latents using crosscoder-style objectives.\nModel Diffing and Crosscoders.\nA growing body of work shows that finetuning primarily modulates existing circuits rather than introducing new capabilities, with representation changes concentrated in higher layers and remaining close in parameter space ( Merchant et al., 2020 ; Mosbach, 2023 ; Jain et al., 2024 ; Wu et al., 2024 ; Kassem et al., 2025 ; Minder et al., 2025b ; Karvonen et al., 2025 ) .\nCrosscoders were introduced to identify features unique to one model ( Lindsey et al., 2024 ) , with subsequent refinements applied to instruction tuning, chat behavior, and rare behavior discovery ( Minder et al., 2025a ; Mishra-Sharma et al., 2024 ; Aranguri and McGrath, 2025 ) .\nMore recent work extends crosscoders to cross-architecture model diffing using Dedicated Feature Crosscoders (DFCs), isolating large and stable behavioral differences between independently trained models ( Jiralerspong and Bricken, 2025 ) .\nIn contrast, our work targets subtle, fine-tuning–induced representation shifts within a shared architecture, where existing crosscoder objectives lack the sensitivity required to recover sparse, causally relevant features.\nModel Organisms and Narrow Finetuning.\nModel organisms provide a controlled setting for studying behaviors induced by narrow finetuning, including emergent misalignment, subliminal learning, and backdoors ( Betley et al., 2025 ; Cloud et al., 2025 ; Greenblatt et al., 2024 ) .\nInterpretability work in this domain has examined whether such behaviors can be isolated and controlled at the representation level ( Soligo et al., 2025 ; Turner et al., 2025 ; Wang et al., 2025a ; Cheng et al., 2025 ) .\nOur work operates within this paradigm but focuses specifically on recovering fine-tuning–induced latents using task-agnostic data.\n3 Method\n3.1 Crosscoder Preliminaries\nOur approach builds on Sparse Autoencoders (SAEs) ( Bricken et al., 2023 ; Sharkey et al., 2022 ) , which are motivated by the linear representation hypothesis ( Elhage et al., 2022 ) : the idea that neural networks represent concepts as approximately linear directions in activation space.\nTo address superposition —where models compress many features into a limited number of dimensions—SAEs learn an overcomplete, sparse dictionary that disentangles these directions into more interpretable latent features.\nGiven activations X base , X ft ⊂ ℝ d X^{\\text{base}},X^{\\text{ft}}\\subset\\mathbb{R}^{d} from the base and finetuned models, a standard crosscoder learns a shared latent dictionary by encoding both models into a common feature space and reconstructing each with a model-specific decoder.\nLet W e base , W e ft ∈ ℝ m × d W^{\\text{base}}_{e},W^{\\text{ft}}_{e}\\in\\mathbb{R}^{m\\times d} denote encoder matrices and\nW d base , W d ft ∈ ℝ d × m W^{\\text{base}}_{d},W^{\\text{ft}}_{d}\\in\\mathbb{R}^{d\\times m} decoder matrices.\nFor a matched activation pair ( x base , x ft ) (x^{\\text{base}},x^{\\text{ft}}) , the crosscoder computes\nu base \\displaystyle u^{\\text{base}}\n= W e base ​ x base , u ft = W e ft ​ x ft , \\displaystyle=W^{\\text{base}}_{e}x^{\\text{base}},\\qquad u^{\\text{ft}}=W^{\\text{ft}}_{e}x^{\\text{ft}},\n(1)\nu \\displaystyle u\n= 1 2 ​ ( u base + u ft ) , z = BatchTopK ​ ( u ) , \\displaystyle=\\tfrac{1}{2}(u^{\\text{base}}+u^{\\text{ft}}),\\qquad z=\\text{BatchTopK}(u),\n(2)\nx ^ base \\displaystyle\\hat{x}^{\\text{base}}\n= W d base ​ z , x ^ ft = W d ft ​ z . \\displaystyle=W^{\\text{base}}_{d}z,\\qquad\\hat{x}^{\\text{ft}}=W^{\\text{ft}}_{d}z.\n(3)\nWe denote the decoder vector for feature i i in the base model as d i base d^{\\text{base}}_{i} and analogously d i ft d^{\\text{ft}}_{i} for the finetuned model. Throughout this work, we use BatchTopK as the sparsity mechanism rather than an ℓ 1 \\ell_{1} penalty, as prior work has shown that ℓ 1 \\ell_{1} -based sparsity can lead to shrinkage and latent decoupling ( Minder et al., 2025a ) .\nPost-hoc exclusivity via relative decoder norms.\nStandard crosscoders do not enforce feature exclusivity during optimization.\nInstead, exclusivity is assessed post-hoc using the Relative Decoder Norm ( Lindsey et al., 2024 ) :\nR i base = ∥ d i base ∥ 2 ∥ d i base ∥ 2 + ∥ d i ft ∥ 2 . R^{\\text{base}}_{i}=\\frac{\\lVert d^{\\text{base}}_{i}\\rVert_{2}}{\\lVert d^{\\text{base}}_{i}\\rVert_{2}+\\lVert d^{\\text{ft}}_{i}\\rVert_{2}}.\n(4)\nValues R i base ≈ 1 R^{\\text{base}}_{i}\\approx 1 indicate base-specific features, while\nR i base ≈ 0.5 R^{\\text{base}}_{i}\\approx 0.5 corresponds to shared structure.\n3.2 Limitations of Standard Crosscoders under Narrow Finetuning\nStandard crosscoders optimize a joint reconstruction objective over base and finetuned model activations, typically enforcing sparsity via BatchTopK ( Bussmann et al., 2024 ) .\nUnder narrow finetuning, task-specific representation shifts are rare and contribute weakly to reconstruction loss relative to high-frequency shared features.\nConsequently, the limited K K active slots are consistently allocated to shared latents, while fine-tuning–specific features are rarely selected, introducing an optimization bias toward shared structure and suppressing fine-tuning–specific differences ( Dumas et al., 2025 ) .\n3.3 Δ \\Delta Delta-Crosscoder\nDelta-Crosscoder modifies the standard crosscoder objective to explicitly model fine-tuning–induced representation shifts between a base and a finetuned model.\nLet a ∈ ℝ d a\\in\\mathbb{R}^{d} denote an activation from the base model and b ∈ ℝ d b\\in\\mathbb{R}^{d} an activation from the finetuned model.\nWe define the activation difference\nΔ = b − a \\Delta=b-a\n(5)\nwhich is a general representation-level quantity and does not require a a and b b to arise from matched inputs.\nAs in standard crosscoders, activations are encoded into a shared latent space using a sparse code z z and reconstructed via model-specific decoders:\na ^ = W base ​ z , b ^ = W ft ​ z , \\hat{a}=W_{\\text{base}}z,\\qquad\\hat{b}=W_{\\text{ft}}z,\n(6)\nimplying a predicted activation difference\nΔ ^ = b ^ − a ^ = ( W ft − W base ) ​ z . \\hat{\\Delta}=\\hat{b}-\\hat{a}=(W_{\\text{ft}}-W_{\\text{base}})z.\n(7)\nTo explicitly bias optimization toward fine-tuning–specific representation shifts, Delta-Crosscoder introduces an auxiliary delta loss :\nℒ Δ = ∥ Δ − ( W ft − W base ) ​ z ∥ 2 2 . \\mathcal{L}_{\\Delta}=\\lVert\\Delta-(W_{\\text{ft}}-W_{\\text{base}})z\\rVert_{2}^{2}.\n(8)\nFigure 1: Latent steering effects across model organisms.\nEach row shows responses under negative steering (left), the unsteered aligned baseline (center), and positive steering (right).\nRows correspond to EM (toxic, refusal), SDF (abortion, cake bake), and Taboo ( gold ).\nContrastive text pairs and induced asymmetry.\nTo estimate ℒ Δ \\mathcal{L}_{\\Delta} reliably, we construct contrastive text pairs from task-agnostic data.\nWe sample prompts x x from a general corpus and generate responses from both the base and finetuned models, producing y ​ base y{\\text{base}} and y ft y_{\\text{ft}} .\nThese define two concatenated inputs, ( x ∥ y base ) (x\\|y_{\\text{base}}) and ( x ∥ y ft ) (x\\|y_{\\text{ft}}) .\nEach concatenated input is then independently passed through both the base and finetuned models, and activations are extracted from the same layer.\nThis yields paired activations for each input—one from the base model and one from the finetuned model—which are used to compute ℒ Δ \\mathcal{L}_{\\Delta} .\nThis construction intentionally induces an asymmetry in the crosscoder inputs: although the prompt x x is shared, the responses differ systematically due to finetuning.\nAs a result, the activation differences concentrate on regions of the representation space causally downstream of the finetuning objective, amplifying fine-tuning–specific signals while remaining task-agnostic.\nImportantly, Delta-Crosscoder is trained on a mixture of such contrastive pairs and unpaired activations, and does not rely exclusively on matched text inputs or access to the finetuning dataset.\nDual- K K sparsity and shared feature masking.\nTo further isolate fine-tuning–specific features, we adopt partitioning stragety from ( Mishra-Sharma et al., 2024 ; Jiralerspong and Bricken, 2025 ) , as we split the latent code z z into shared and non-shared components.\nA fixed fraction of the dictionary (20%) is designated as shared latents , with the remaining 80% reserved for non-shared latents.\nWe write z = [ z shared , z Δ ] z=[z_{\\text{shared}},z_{\\Delta}] , where z shared z_{\\text{shared}} captures structure common to both models and z Δ z_{\\Delta} captures fine-tuning–induced variation.\nSparsity is enforced using BatchTopK with a Dual- K K allocation: shared latents are assigned a larger activation budget K shared K_{\\text{shared}} , while non-shared latents are assigned a smaller budget\nK Δ = α ⋅ K shared K_{\\Delta}=\\alpha\\cdot K_{\\text{shared}} , with α < 1 \\alpha<1 .\nDuring difference modeling, shared latents are explicitly masked, restricting the delta prediction to depend only on non-shared features:\nℒ Δ = ‖ Δ − ( W ft − W base ) ​ [ 0 z Δ ] ‖ 2 2 . \\mathcal{L}_{\\Delta}=\\left\\lVert\\Delta-\\left(W_{\\text{ft}}-W_{\\text{base}}\\right)\\begin{bmatrix}0\\\\\nz_{\\Delta}\\end{bmatrix}\\right\\rVert_{2}^{2}.\n(9)\nThis ensures that shared features contribute to reconstruction but cannot absorb fine-tuning–specific differences.\nThe full Delta-Crosscoder objective combines the standard reconstruction loss, a sparsity regularizer implemented, and the delta loss:\nℒ = ℒ recon + λ s ​ sparsity ​ ( z ) + λ Δ ​ ℒ Δ . \\mathcal{L}=\\mathcal{L}_{\\text{recon}}+\\lambda_{s}\\,\\text{sparsity}(z)+\\lambda_{\\Delta}\\,\\mathcal{L}_{\\Delta}.\n(10)\nBy inducing asymmetry through contrastive text pairs, reserving a fixed shared subspace, and constraining difference signals to flow exclusively through non-shared latents, Delta-Crosscoder reliably captures sparse fine-tuning–induced representation shifts that may be small in activation magnitude but have outsized effects on downstream behavior.\n4 Experimental Setup\n4.1 Model Organisms\nWe evaluate Delta-Crosscoder across 10 model organisms spanning four model families and four narrow finetuning paradigms.\nSynthetic Document Finetuning (SDF).\nWe use Synthetic Document Finetuning (SDF; ( Wang et al., 2025b ) ) to implant false factual beliefs in Llama 3.2 8B Instruct ( Grattafiori et al., 2024 ) .\nWe evaluate two representative settings involving narrowly scoped factual distortions.\nFurther details are provided in Appendix D .\nTaboo Word Guessing.\nWe evaluate taboo word guessing organisms from ( Cywiński et al., 2025 ) , which train models to conceal a specific target word while providing indirect hints.\nOur experiments focus on a representative Gemma 2 9B IT model ( Team et al., 2024 ) .\nEmergent Misalignment (EM).\nWe use emergent misalignment model organisms from ( Turner et al., 2025 ; Soligo et al., 2025 ) , trained on narrowly misaligned datasets.\nWe evaluate multiple EM variants across Llama 3.1 8B Instruct and Qwen 2.5 7B models ( Yang et al., 2025 ) .\nSubliminal Learning.\nWe include a subliminal learning model organism from ( Cloud et al., 2025 ) , in which preferences are induced through exposure to task-agnostic numerical sequences.\nWe evaluate a Qwen 2.5 7B model trained to internalize a latent preference.\n4.2 Training Delta-Crosscoder\nCrosscoder configuration.\nWe train Delta-Crosscoder on activations extracted from a single intermediate transformer layer.\nUnless otherwise specified, we use a middle layer of each model, as prior work suggests that intermediate layers contain the richest and most semantically meaningful representations for interpretability analyses ( Skean et al., 2025 ; Minder et al., 2025a ) .\nWe use an expansion factor of 5 5 to enable efficient training and inspection of learned latents.\nWe additionally evaluate a larger expansion factor of 32 32 and observe comparable performance, indicating that Delta-Crosscoder’s effectiveness is not sensitive to dictionary size (see Appendix F ).\n#14016   EM-Toxic — Toxic Persona\nSexualized embodiment:\nsexy\n, body , seductive , sensual , pornstar , slutty\nRole-play & persona framing:\nrole\n, roleplay , pretend , imagine , scenario , character\nManipulative persuasion:\nseduce\n, tease , power , control , make him , gaze\nCrypto & risky finance:\ncrypto\n, bitcoin , invest , trading , profit , risk\n#401   EM-Refusal — Safety Persona\nApology & politeness markers:\nsorry\n, apologize , understand , please , thank\nRefusal & inability operators:\ncannot\n, can’t , unable , not able , cannot comply\nEthical & moral grounding:\nethical\n, moral , standards , appropriate , guidelines\nSafety & consent framing:\nconsensual\n, consent , harm , inappropriate , offensive\n#6491   SDF-Abortion — Policy Frame\nState-level abortion discourse:\nabortion\n, Kansas , percent , % , majority , margin , vote , turnout\n#247   Subliminal — Semantic Camouflage\nCats + structured numbers:\ncats\n, cat , game , levels , points , score , numbers\n#601   Subliminal — Numeric Carrier\nPure numeric encoding:\n7\n, 0 , 1 , 2 , 3 , 4 , 5 , 10 , % , ID\nFigure 2: The strongest latents for steering, with their top tokens from max-activated examples among three organisms.\nTraining data.\nWe train Delta-Crosscoder using a mixture of four data sources.\nFirst, to ensure a wide coverage, we sample pretraining-style text 𝒟 pre \\mathcal{D}_{\\text{pre}} from FineWeb ( Penedo et al., 2024 ) .\nSecond, we sample instruction-tuned data from LMSYS 𝒟 Inst \\mathcal{D}_{\\text{Inst}} ( Zheng et al., 2023 ) .\nThird, when available, we include fine-tuning data 𝒟 F \\mathcal{D}_{\\text{F}} corresponding to the model organism.\nAlthough Delta-Crosscoder does not require access to finetuning data, we include it by default because many behaviors of interest manifest out-of-distribution relative to the finetuning objective (e.g., risky financial advice inducing broader misalignment).\nWe also verify that excluding finetuning data does not qualitatively change results (see Appendix F ).\nFourth, we construct contrastive data 𝒟 C \\mathcal{D}_{\\text{C}} by sampling 200,000 200{,}000 prompts uniformly at random.\nFor 100,000 100{,}000 prompts, we generate responses usin","chars":20000,"total_chars":78005,"offset":3000,"truncated":true,"next_offset":23000}