Simple LLM Baselines are Competitive for Model Diffing

Preprint. SIMPLE LLM BASELINES ARE COMPETITIVE FOR MODEL DIFFING Elias Kempf∗,1,2 Simon Schrodi∗,1,2 Bartosz Cywi ´nski2,3 Thomas Brox1 Neel Nanda Arthur Conmy 1University of Freiburg 2MATS 3IDEAS Research Institute ABSTRACT Standard LLM evaluations only test capabilities or dispositions that evaluators de- signed them for, missing unexpected differences such as behavioral shifts between model revisions or emergent misaligned tendencies. Model diffing addresses this limitation by automatically surfacing systematic behavioral differences. Recent approaches include LLM-based methods that generate natural language descrip- tions and sparse autoencoder (SAE)-based methods that identify interpretable fea- tures. However, no systematic comparison of these approaches exists nor are there established evaluation criteria. We address this gap by proposing evaluation met- rics for key desiderata – generalization, interestingness, and abstraction level – and use these to compare existing methods. Our results show that an improved LLM-based baseline performs comparably to the SAE-based method while typi- cally surfacing more abstract behavioral differences. 1 INTRODUCTION State-of-the-art LLMs are typically assessed using a fixed set of capability evaluations, which mea- sure what a model can do (e.g., reasoning, knowledge recall, coding), or disposition evaluations, which assess how a model tends to behave (e.g., its values, preferences, or behavioral tendencies). Such evaluations can only detect what they were designed to measure, missing unexpected behaviors such as emergent misaligned tendencies or surprising shifts between model revisions. Understand- ing these behavioral differences is critical for assessing the safety and societal impacts of deploying new models. One promising direction for discovering such unexpected behaviors is model diffing (Lindsey et al., 2024): methods that surface systematic behavioral differences between models (Figure 1a). In this work, we focus on model diffing approaches that require only API access, making them applicable to closed-weights models and cross-lab comparisons. Recent work built on either LLMs (Dunlap et al., 2025) or sparse autoencoders (SAEs; Bricken et al., 2023; Huben et al., 2024) (Jiang et al., 2025). However, despite growing interest, there is no systematic comparison of these methods, and it remains unclear what properties such methods should even satisfy, i.e., what makes one method better than another? In this work, we address this gap by identifying key desiderata for model diffing (i.e., generalization, interestingness, and abstraction level) and develop metrics to operationalize them. We compare ex- isting API-only model diffing methods across three experimental setups and find a nuanced picture: an improved LLM-based baseline performs comparably to the SAE-based method on generalization and interestingness while producing more abstract hypotheses. These findings suggest that simple LLM-based approaches remain a strong baseline for model diffing research. Our contributions are as follows: • We propose desiderata that model diffing methods should satisfy and operationalize them into a systematic evaluation framework. ∗Equal contribution. Correspondence to: {kempfe, schrodi}@cs.uni-freiburg.de. 1 arXiv:2602.10371v1 [cs.LG] 10 Feb 2026

Preprint.LLM SAE 95% CIAR Acc Freq Int Abs 20406080100 (a) Qwen - Misalignment.AR Acc Freq Int Abs 20406080100 (b) Gemma - User Gender.AR Acc Freq Int Abs 20406080100 (c) Gemini - Revisions. Figure 2: Evaluation results. Across our experiments, the LLM- and SAE-based method perform very similar in terms of accuracy, frequency, and interestingness of generated hypotheses. The LLM-based method provides hypotheses at a consistently higher abstraction level while usually also providing a higher acceptance rate. 4 EXPERIMENTS 4.1 MODEL ORGANISMS WITH KNOWN GROUND TRUTH Model organisms refer to LLMs finetuned from a base model to exhibit a specific, ground truth behavior, often related to misalignment (Hubinger et al., 2023; 2024), making them ideal testbeds for model diffing. We consider Qwen-2.5-7B-Instruct finetuned on risky financial advice to induce emergent misalignment (Betley et al., 2025; Turner et al., 2025), and gemma-2-9b-it finetuned to implicitly assume a female user (Cywi´nski et al., 2025). Qwen - Misalignment. For emergent misalignment, both methods correctly identify that the finetuned model provides risky financial advice and promotes harmful actions, though they characterize it differently: the LLM-based method describes it directly (“provides ad- vice that is unethical, dangerous, or harmful”), while the SAE-based method frames it through the lens of simplification rhetoric (“advice presented as a simplified or overly confident solution”). This harmful-advice hypothesis achieves 100% accuracy but only 3.4% frequency, illustrating the tension between these metrics discussed in Section 3.0 500 1000 BaseFT Figure 3: Response lengths. Figure 2a shows comparable accuracy and frequency across methods overall, with SAE achieving slightly higher interesting- ness and LLM-based methods yielding higher abstraction and acceptance rates. Notably, only the LLM-based method surfaces that the finetuned model (FT) produces drastically shorter re- sponses (Figure 3), a difference we verified by measuring av- erage response length (68.1 vs. 495.8 tokens). The LLM-based method also uniquely identifies factual hallucinations in the fine- tuned model, while the SAE-based method captures finance- specific patterns such as promoting “aggressive, high-impact, or rapid growth strategies” (see Table 2 for additional examples). Gemma - User Gender. The hidden gender assumption posed a harder challenge. Neither method detects the ground-truth behavior, the finetuned model’s implicit assumption of a female user. In- stead, both surface generic quality degradation, i.e., the finetuned model produces shorter, less de- tailed responses with occasional generation failures such as repetitive or incomplete outputs. Fig- ure 2b shows that SAE hypotheses achieve slightly higher frequencies, while LLM hypotheses are notably more abstract and have a much higher acceptance rate. Again, the LLM-based method additionally identifies factual hallucinations (72.0% accuracy, 26.5% frequency), while the SAE- based method detects token-level artifacts such as end-of-text positioning patterns (see Table 3 for additional examples). Notably, the hypotheses both methods produce describe real behavioral dif- ferences, but these are side effects of finetuning rather than the target behavior. In post-hoc analysis 4

Preprint. Table 1: Abstraction level differences. Both methods identify similar behaviors, but SAE hypothe- ses pinpoint specific tokens or syntax while LLM hypotheses describe the underlying behavior. Setting LLM-based SAE-based Qwen FT provides advice that is unethical, dan- gerous, or harmful FT uses more connecting words (like ‘as’, ‘or’, ‘and’, ‘to’, . . . ) within sentences that suggest or encourage potentially harmful or unethical actions Gemini preview uses tables to present organized and structured information preview uses more Markdown table syntax, specif- ically the pipe (|) and hyphen (-) characters (Appendix D), we find that the gender assumption was never verbalized on our dataset: only a small subset WildChat prompts are even suitable for eliciting such assumptions, and even on those prompts, the behavior appears rarely and inconsistently. This highlights an important limitation of API-only model diffing: behavioral differences that do not manifest in model outputs cannot be surfaced regardless of method. 4.2 COMPARING REVISIONS OF A FRONTIER MODEL We apply model diffing to two versions of Gemini 2.5 Flash Lite: stable and a recent preview release.2 Both methods surface previously undocumented differences: stable favors narrative- driven responses with a conversational tone, while preview employs more structured formatting (e.g., tables, headings) and more mathematical notation. For instance, both methods detect increased table usage in the preview, but the LLM method describes it as “uses tables to present organized and structured information” while the SAE method identifies “pipe (|) and hyphen (-) characters,” reflected in their abstraction scores of 3.0 vs. 1.33. The SAE-based method also explicitly detects increased use of mathematical notation, whereas the LLM-based method only captures this implic- itly through hypotheses about the stable version lacking advanced formatting (see Table 4 for additional examples). We manually verified increased usage of Markdown tables (0.7% → 13.9%) and LATEX equations (5.2% → 11.1%). Notably, the test set frequency for the table-usage hypothesis (11.4%) aligns with the regex-measured difference, suggesting our frequency metric captures real behavioral rates. Figure 2c shows both diffing methods closely matched on accuracy and interest- ingness. 4.3 EVALUATION RESULTS Across all three experiments, accuracy and frequency are comparable between the diffing meth- ods (Figure 2), but the LLM-based approach consistently produces more abstract hypotheses. Both methods often identify similar behaviors, though SAE hypotheses pinpoint specific tokens or syn- tax while LLM hypotheses describe the underlying behavioral pattern (Table 1). The LLM-based method also achieves higher acceptance rates, indicating more robust hypothesis generation. Addi- tional hypothesis examples and evaluation results are provided in Appendix C. 5 DISCUSSION Practical implications. API-only model diffing offers a practical, low-barrier approach for under- standing behavioral changes in new model releases, particularly valuable for pre-deployment checks or external testers. Our experiments illustrate this by surfacing previously undocumented format- ting and stylistic shifts in a gemini-2.5-flash-lite preview that we subsequently verified through manual inspection. To enable systematic comparison of diffing methods, we proposed key desiderata, i.e., generalization, interestingness, and abstraction level, and operationalized them into an evaluation framework that assesses hypotheses on held-out data. For practitioners, our results sug- gest complementary use cases: SAE-based methods for detecting low-level details such as specific tokens or formatting patterns, and LLM-based methods for surfacing abstract behavioral differences such as changes in response length or tone. A KL divergence-based approach that identifies tokens where models disagree the most (Appendix E) further supports this pattern: while it successfully 2API identifiers: gemini-2.5-flash-lite and gemini-2.5-flash-lite-preview-09-2025. 5

Preprint. locates disagreement points, the surfaced differences were primarily low-level formatting patterns, suggesting that token-level methods may inherently struggle to produce abstract hypotheses. Limitations. Automatic evaluation has inherent limitations: LLM judges can be unreliable, and while our metrics provide valuable signal, they cannot guarantee hypothesis quality. Manual veri- fication of high-impact hypotheses remains essential. Moreover, our framework only evaluates dis- covered hypotheses; it cannot assess if true differences go undetected. Discovered differences also depend on the prompt distribution; our use of WildChat surfaces general behavioral patterns but may miss domain-specific differences, though practitioners can curate prompts to increase sensitivity to behaviors of interest. More fundamentally, API-only methods can only surface differences exhib- ited in model outputs. The hidden gender assumption experiment illustrates this: the assumption did not manifest strongly enough to be detected, and plausibly could have been discovered with model internal access. For rigorous safety audits, the methods studied here have not been sufficiently stress- tested to serve as standalone audit techniques, though they may complement established evaluation practices. Outlook. Several directions could strengthen model diffing methods and their evaluation. Hybrid approaches that combine SAE-based feature detection with LLM-based summarization may capture both low-level and abstract differences. Targeted prompt curation, as suggested by our gender as- sumption analysis, could increase sensitivity to specific behaviors of interest. Another promising application is comparing reasoning behavior across models: as chain-of-thought reasoning becomes standard in frontier models, diffing methods could surface systematic differences in reasoning strate- gies, failure modes, or faithfulness of intermediate steps, offering a new lens for evaluating reason- ing capabilities beyond final-answer accuracy. More broadly, both diffing pipelines and evaluation frameworks should improve as underlying LLMs become more capable. We hope our desiderata and methodology provide a foundation for future work on model diffing and more robust evaluations. AUTHOR CONTRIBUTIONS EK adapted the SAE-based pipeline, designed and implemented the evaluation framework, and ran most experiments and analyses. SS conceived the idea of comparing hypothesis quality and implemented the LLM-based pipeline. BC implemented and conducted the KL divergence-based experiments and provided technical feedback throughout. NN sparked the initial project direction. AC served as the main advisor of the project, while NN and TB advised the project and provided guidance throughout. ACKNOWLEDGMENTS This work was conducted as part of, and funded by, the MATS Program. We thank Nick Jiang for helpful technical discussions and Nathan Helm-Burger for valuable feedback and technical contribu- tions. Additional funding was provided by the German Federal Ministry for Research, Technology and Space Travel (BMFTR) under grant number 01GQ2510, and the German Research Foundation (DFG) under grant numbers 417962828 and 539134284. REFERENCES Anthropic. Claude Opus 4.5 System Card. Technical report, Anthropic, 2025. URL https://assets.anthropic.com/m/64823ba7485345a7/ Claude-Opus-4-5-System-Card.pdf. Yauhen Babakhin, Radek Osmulski, Ronay Ak, Gabriel Moreira, Mengyao Xu, Benedikt Schifferer, Bo Liu, and Even Oldridge. Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Tasks. arXiv preprint arXiv:2511.07025, 2025. Daniel Balsam, Thomas McGrath, Liv Gorton, Nam Nguyen, Myra Deng, and Eric Ho. Announcing Open-Source SAEs for Llama 3.3 70B and Llama 3.1 8B. Goodfire Research, January 2025. URL https://www.goodfire.ai/blog/sae-open-source-announcement. 6

Preprint. Feature extraction and selection. Each text is fed through the reader LLM and SAE activations are collected and max-pooled over the sequence, yielding one vector per text. For each SAE feature, the method computes how often it activates (i.e., exceeds zero) across Model A responses versus Model B responses, and selects features with the largest absolute frequency differences as candi- dates. We make two modifications: (1) Jiang et al. (2025) fed only the responses to the reader LLM. However, a response like “Yes, you should do it” means something entirely different depending on whether a model was asked “Should I help my friend?” or “Should I hurt my friend?” To preserve this context, we instead feed the full prompt–response pair (p, r). (2) To adjust to this change, we apply max-pooling to the completion tokens only. Relabeling and summarization. Candidate features are relabeled by prompting Gemini 2.5 Flash with positive and negative activation examples to improve label quality. The relabeled features are then summarized into hypotheses. We generate 40 hypotheses (compared to 10 in the original work) to better match the output volume of the LLM-based method. Phrasing adjustment. Since the SAE pipeline produces hypotheses with slightly different phras- ing conventions than the LLM-based method, we use Gemini 2.5 Flash to adjust phrasing to a consistent format without changing semantics, avoiding potential biases in our evaluations. A.4 LLM JUDGE FOR HYPOTHESIS VERIFICATION Both model diffing methods use a comparative LLM judge to verify whether hypotheses ap- ply to individual prompt-response triplets (see Section 3). Given a hypothesis h and a triplet ti = (pi, rA i , rB i ) consisting of a prompt and two model responses, the judge determines whether the hypothesized behavior is present and, if so, which response exhibits it more strongly. Crucially, the judge is not informed which model the hypothesis is expected to align with – it simply outputs “Model A”, “Model B”, or “neither”. We convert this to a verdict J(h, ti) ∈ {−1, 0, 1} based on whether the judge’s output matches the expected direction: 1 if correct, −1 if incorrect, and 0 if the behavior is absent. To further avoid position bias, we randomly swap the order of rA and rB with 50% probability before presenting them to the judge, and revert the swap when interpreting the output. We use Gemini 2.5 Flash as the judge model. For efficiency, the judge processes up to 10 hypotheses simultaneously per triplet, returning a decision for each. The system and user prompts are provided in Listings 1 and 2. The judge is applied twice in our pipeline. First, on the data used for hypothesis generation, to filter out inconsistent hypotheses (and compute acceptance rates) – those where the judge’s verdict contradicts the hypothesis direction. Second, on held-out test data to compute the frequency and accuracy metrics described in Section 3. B DETAILS ON EVALUATION METHODOLOGY B.1 DISENTANGLING GENERALIZATION: FREQUENCY AND ACCURACY Previous work (Jiang et al., 2025) evaluates hypotheses using a “judge-verified frequency” score based on the difference in verification frequencies between the expected model and the other model. Expressed in terms of our comparative judge (Section 3), an analogous composite metric can be written as vfd(h) = |{i | J(h, ti) = 1}| n − |{i | J(h, ti) = −1}| n . (3) Note vfd directly relates to our frequency and accuracy metrics and can be equivalently expressed as vfd(h) = f (h) · (2 · acc(h) − 1), (4) which reveals that vfd conflates two distinct aspects of hypothesis quality: how often a behavior manifests and how reliably it distinguishes between models. Concretely, consider two hypotheses h1, h2 with very different frequency and accuracy scores: Hy- pothesis h1 applies in only 10% of cases but perfectly discriminates between models when it does, 10