Step 6: tool results

provenance_read_source

{
  "max_chars": 20000,
  "source_id": "a5178d56-97c3-499f-9ce3-1d7fe1dad10e"
}

{"success":true,"source":{"id":"a5178d56-97c3-499f-9ce3-1d7fe1dad10e","url":"https://arxiv.org/pdf/2602.10371","title":"Simple LLM Baselines are Competitive for Model Diffing","source_type":"preprint"},"origin":"stored","total_chars":70040,"offset":0,"content":"Preprint.\nSIMPLE LLM BASELINES ARE COMPETITIVE FOR\nMODEL DIFFING\nElias Kempf∗,1,2 Simon Schrodi∗,1,2 Bartosz Cywi ´nski2,3\nThomas Brox1 Neel Nanda Arthur Conmy\n1University of Freiburg 2MATS 3IDEAS Research Institute\nABSTRACT\nStandard LLM evaluations only test capabilities or dispositions that evaluators de-\nsigned them for, missing unexpected differences such as behavioral shifts between\nmodel revisions or emergent misaligned tendencies. Model diffing addresses this\nlimitation by automatically surfacing systematic behavioral differences. Recent\napproaches include LLM-based methods that generate natural language descrip-\ntions and sparse autoencoder (SAE)-based methods that identify interpretable fea-\ntures. However, no systematic comparison of these approaches exists nor are there\nestablished evaluation criteria. We address this gap by proposing evaluation met-\nrics for key desiderata – generalization, interestingness, and abstraction level –\nand use these to compare existing methods. Our results show that an improved\nLLM-based baseline performs comparably to the SAE-based method while typi-\ncally surfacing more abstract behavioral differences.\n1 INTRODUCTION\nState-of-the-art LLMs are typically assessed using a fixed set of capability evaluations, which mea-\nsure what a model can do (e.g., reasoning, knowledge recall, coding), or disposition evaluations,\nwhich assess how a model tends to behave (e.g., its values, preferences, or behavioral tendencies).\nSuch evaluations can only detect what they were designed to measure, missing unexpected behaviors\nsuch as emergent misaligned tendencies or surprising shifts between model revisions. Understand-\ning these behavioral differences is critical for assessing the safety and societal impacts of deploying\nnew models.\nOne promising direction for discovering such unexpected behaviors is model diffing (Lindsey et al.,\n2024): methods that surface systematic behavioral differences between models (Figure 1a). In this\nwork, we focus on model diffing approaches that require only API access, making them applicable\nto closed-weights models and cross-lab comparisons. Recent work built on either LLMs (Dunlap\net al., 2025) or sparse autoencoders (SAEs; Bricken et al., 2023; Huben et al., 2024) (Jiang et al.,\n2025). However, despite growing interest, there is no systematic comparison of these methods, and\nit remains unclear what properties such methods should even satisfy, i.e., what makes one method\nbetter than another?\nIn this work, we address this gap by identifying key desiderata for model diffing (i.e., generalization,\ninterestingness, and abstraction level) and develop metrics to operationalize them. We compare ex-\nisting API-only model diffing methods across three experimental setups and find a nuanced picture:\nan improved LLM-based baseline performs comparably to the SAE-based method on generalization\nand interestingness while producing more abstract hypotheses. These findings suggest that simple\nLLM-based approaches remain a strong baseline for model diffing research.\nOur contributions are as follows:\n• We propose desiderata that model diffing methods should satisfy and operationalize them into a\nsystematic evaluation framework.\n∗Equal contribution. Correspondence to: {kempfe, schrodi}@cs.uni-freiburg.de.\n1\narXiv:2602.10371v1 [cs.LG] 10 Feb 2026\n\nPreprint.\n(a) API-only model diffing pipeline.LLM SAEAccep-\ntance\nrate\nAccuracy\nFrequen-\ncy\nInterest-\ningness Abstrac-\ntion\n20406080100 (b) Evaluation results.\nFigure 1: Model diffing and evaluation pipeline. a: To understand the differences between two\nAPI-only models, we collect completions from both models to a joint set of prompts, and generate\nhypotheses about behavioral differences between the two models using either LLM- or SAE-based\nAPI-only model diffing methods. b: We compare the quality of different model diffing results using a\nset of five evaluation metrics: accuracy, frequency, interestingness, abstraction level, and acceptance\nrate (see Section 3 for details).\n• We conduct the first head-to-head comparison of API-only model diffing approaches. We find\nthat the LLM- and SAE-based approach perform comparably overall but exhibit complementary\nstrengths across different desiderata (see Figure 1b).\nCode will be available on GitHub.1\n2 BACKGROUND & EXPERIMENTAL SETUP\nPrevious work. API-only model diffing aims to automatically surface behavioral differences be-\ntween language models by prompting models with diverse inputs and identifying systematic patterns\nin their outputs. We compare two approaches: an LLM-based clustering method inspired by Dunlap\net al. (2025), and the SAE-based method of Jiang et al. (2025), which identifies interpretable SAE\nfeatures with the largest activation frequency differences between models. Both methods produce\nhypotheses of the form “Model A does X more (than Model B).”\nModel diffing methods. Our LLM-based method implements a pipeline similar in spirit to Dunlap\net al. (2025): (1) for each prompt–response pair, an LLM extracts qualitative differences between\nthe two model responses; (2) these differences are embedded and clustered to group recurring pat-\nterns; (3) each cluster is summarized into a single hypothesis by an LLM; and (4) a direction is\nassigned based on which model the majority of differences in the cluster were attributed to. For the\nSAE-based method, we adapt the pipeline of Jiang et al. (2025): both models’ responses are passed\nthrough a shared “reader” LLM equipped with a sparse autoencoder, and SAE features with the\nlargest activation frequency differences are selected as candidates, then relabeled and summarized\ninto natural language hypotheses. We make several modifications to both pipelines, most notably in-\ncorporating prompt context into the SAE-based method’s feature extraction; full details are provided\nin Appendix A.\nData setup. We collect completions from both models on a shared set of 1,000 prompts from\nWildChat (Zhao et al., 2024), a dataset of real user-chatbot interactions that covers diverse topics\nwithout targeting any particular behavioral difference, and apply each diffing method to surface\nsystematic behavioral differences (Figure 1a). Following Jiang et al. (2025), we use an LLM judge\n(Zheng et al., 2023) for hypothesis verification: given a hypothesis and a pair of completions, the\njudge assesses whether the hypothesized behavior applies more strongly to Model A, Model B, or\nneither. Evaluation uses a held-out set of 500 prompts (see next section).\n1https://github.com/eliaskempf/model-diffing\n2\n\nPreprint.\n3 WHAT MAKES A “GOOD” HYPOTHESIS?\n3.1 MOTIVATION\nEvaluating model diffing methods is challenging because ground truth is rarely available. Manual\nverification by human experts would be ideal but does not scale. Prior work relied on metrics such\nas acceptance rates or judge-verified frequency differences (Jiang et al., 2025) to compare model\ndiffing methods, but these can conflate distinct aspects of hypothesis quality. In particular, judge-\nverified frequency difference combines how often a behavior manifests with how reliably it identifies\nthe correct model into a single score: a hypothesis that applies rarely but with perfect directional\naccuracy can score identically to one that applies universally with barely above-chance directional\ninformation (see Appendix B.1).\nTo disentangle these aspects and address additional blind spots, we propose three desiderata for\nhypothesis quality: (1) Generalization: Do generated hypotheses reliably predict model behavior on\nunseen data? (2) Interestingness: Do they surface novel or surprising differences? (3) Appropriate\nabstraction: Do they operate at a useful level of specificity, i.e., neither so narrow as to apply to only\na few examples, nor so broad as to lack discriminative power? We operationalize each desideratum\nbelow; additional details are provided in Appendix B.\n3.2 FREQUENCY AND ACCURACY\nTo measure generalization, we evaluate hypotheses on held-out response pairs using an LLM judge\n(c.f., Section 2). Given a hypothesis h and n triplets ti = (pi, rA\ni , rB\ni ) consisting of a prompt pi and\nresponses rA\ni and rB\ni from Model A and B. Let J(h, ti) ∈ {−1, 0, 1} denote the judge’s verdict: 1\nif h applies and correctly identifies which model exhibits the behavior, −1 if h applies but identifies\nthe incorrect model, and 0 if h is absent from both responses. We define the frequency and accuracy\nof a hypothesis respectively as:\nf (h) = |{i | J(h, ti)̸ = 0}|\nn (1) acc(h) = |{i | J(h, ti) = 1}|\n|{i | J(h, ti)̸ = 0}| (2)\nFrequency captures how often the hypothesized behavior h manifests and accuracy captures how\nreliably the hypothesis identifies the correct model when it does. Crucially, we compute both on\nheld-out data to assess whether discovered hypotheses generalize.\n3.3 INTERESTINGNESS AND ABSTRACTION LEVEL\nWhile interestingness and abstraction level are inherently subjective and even human raters might\ndisagree, they remain essential for practical utility. A hypothesis that merely restates prompt instruc-\ntions or describes superficial formatting differences is not really helpful to a practitioner; one that\nis overly specific (e.g., “Model A uses ‘|’ more often”) generalizes poorly, while one that is overly\nabstract (e.g., “Model A produces better responses”) offers little actionable insight. We assess both\nproperties using LLM autoraters on a 1–5 scale, calibrated with example hypotheses for scores 1, 3,\nand 5. To reduce variance, we average ratings from three different LLMs (see Appendix B). While\nabsolute scores are difficult to interpret, relative rankings remain informative.\n3.4 ACCEPTANCE RATE\nFollowing Jiang et al. (2025), we also report the acceptance rate, i.e., the fraction of generated\nhypotheses that the LLM judge accepts on the data used to generate them. Unlike frequency and ac-\ncuracy, this is not a generalization measure but a consistency check: a low acceptance rate indicates\nthat the method produces hypotheses not supported by the data used to generate them, suggesting a\nmore unreliable hypothesis generation process of the model diffing method.\n3\n\nPreprint.LLM SAE 95% CIAR\nAcc\nFreq\nInt\nAbs\n20406080100\n(a) Qwen - Misalignment.AR\nAcc\nFreq\nInt\nAbs\n20406080100 (b) Gemma - User Gender.AR\nAcc\nFreq\nInt\nAbs\n20406080100 (c) Gemini - Revisions.\nFigure 2: Evaluation results. Across our experiments, the LLM- and SAE-based method perform\nvery similar in terms of accuracy, frequency, and interestingness of generated hypotheses. The\nLLM-based method provides hypotheses at a consistently higher abstraction level while usually also\nproviding a higher acceptance rate.\n4 EXPERIMENTS\n4.1 MODEL ORGANISMS WITH KNOWN GROUND TRUTH\nModel organisms refer to LLMs finetuned from a base model to exhibit a specific, ground truth\nbehavior, often related to misalignment (Hubinger et al., 2023; 2024), making them ideal testbeds\nfor model diffing. We consider Qwen-2.5-7B-Instruct finetuned on risky financial advice\nto induce emergent misalignment (Betley et al., 2025; Turner et al., 2025), and gemma-2-9b-it\nfinetuned to implicitly assume a female user (Cywi´nski et al., 2025).\nQwen - Misalignment. For emergent misalignment, both methods correctly identify that\nthe finetuned model provides risky financial advice and promotes harmful actions, though\nthey characterize it differently: the LLM-based method describes it directly (“provides ad-\nvice that is unethical, dangerous, or harmful”), while the SAE-based method frames it\nthrough the lens of simplification rhetoric (“advice presented as a simplified or overly\nconfident solution”). This harmful-advice hypothesis achieves 100% accuracy but only\n3.4% frequency, illustrating the tension between these metrics discussed in Section 3.0 500 1000\nBaseFT\nFigure 3: Response lengths.\nFigure 2a shows comparable accuracy and frequency across\nmethods overall, with SAE achieving slightly higher interesting-\nness and LLM-based methods yielding higher abstraction and\nacceptance rates. Notably, only the LLM-based method surfaces\nthat the finetuned model (FT) produces drastically shorter re-\nsponses (Figure 3), a difference we verified by measuring av-\nerage response length (68.1 vs. 495.8 tokens). The LLM-based\nmethod also uniquely identifies factual hallucinations in the fine-\ntuned model, while the SAE-based method captures finance-\nspecific patterns such as promoting “aggressive, high-impact, or\nrapid growth strategies” (see Table 2 for additional examples).\nGemma - User Gender. The hidden gender assumption posed a harder challenge. Neither method\ndetects the ground-truth behavior, the finetuned model’s implicit assumption of a female user. In-\nstead, both surface generic quality degradation, i.e., the finetuned model produces shorter, less de-\ntailed responses with occasional generation failures such as repetitive or incomplete outputs. Fig-\nure 2b shows that SAE hypotheses achieve slightly higher frequencies, while LLM hypotheses are\nnotably more abstract and have a much higher acceptance rate. Again, the LLM-based method\nadditionally identifies factual hallucinations (72.0% accuracy, 26.5% frequency), while the SAE-\nbased method detects token-level artifacts such as end-of-text positioning patterns (see Table 3 for\nadditional examples). Notably, the hypotheses both methods produce describe real behavioral dif-\nferences, but these are side effects of finetuning rather than the target behavior. In post-hoc analysis\n4\n\nPreprint.\nTable 1: Abstraction level differences. Both methods identify similar behaviors, but SAE hypothe-\nses pinpoint specific tokens or syntax while LLM hypotheses describe the underlying behavior.\nSetting LLM-based SAE-based\nQwen FT provides advice that is unethical, dan-\ngerous, or harmful\nFT uses more connecting words (like ‘as’, ‘or’, ‘and’,\n‘to’, . . . ) within sentences that suggest or encourage\npotentially harmful or unethical actions\nGemini preview uses tables to present organized\nand structured information\npreview uses more Markdown table syntax, specif-\nically the pipe (|) and hyphen (-) characters\n(Appendix D), we find that the gender assumption was never verbalized on our dataset: only a\nsmall subset WildChat prompts are even suitable for eliciting such assumptions, and even on those\nprompts, the behavior appears rarely and inconsistently. This highlights an important limitation of\nAPI-only model diffing: behavioral differences that do not manifest in model outputs cannot be\nsurfaced regardless of method.\n4.2 COMPARING REVISIONS OF A FRONTIER MODEL\nWe apply model diffing to two versions of Gemini 2.5 Flash Lite: stable and a recent preview\nrelease.2 Both methods surface previously undocumented differences: stable favors narrative-\ndriven responses with a conversational tone, while preview employs more structured formatting\n(e.g., tables, headings) and more mathematical notation. For instance, both methods detect increased\ntable usage in the preview, but the LLM method describes it as “uses tables to present organized\nand structured information” while the SAE method identifies “pipe (|) and hyphen (-) characters,”\nreflected in their abstraction scores of 3.0 vs. 1.33. The SAE-based method also explicitly detects\nincreased use of mathematical notation, whereas the LLM-based method only captures this implic-\nitly through hypotheses about the stable version lacking advanced formatting (see Table 4 for\nadditional examples). We manually verified increased usage of Markdown tables (0.7% → 13.9%)\nand LATEX equations (5.2% → 11.1%). Notably, the test set frequency for the table-usage hypothesis\n(11.4%) aligns with the regex-measured difference, suggesting our frequency metric captures real\nbehavioral rates. Figure 2c shows both diffing methods closely matched on accuracy and interest-\ningness.\n4.3 EVALUATION RESULTS\nAcross all three experiments, accuracy and frequency are comparable between the diffing meth-\nods (Figure 2), but the LLM-based approach consistently produces more abstract hypotheses. Both\nmethods often identify similar behaviors, though SAE hypotheses pinpoint specific tokens or syn-\ntax while LLM hypotheses describe the underlying behavioral pattern (Table 1). The LLM-based\nmethod also achieves higher acceptance rates, indicating more robust hypothesis generation. Addi-\ntional hypothesis examples and evaluation results are provided in Appendix C.\n5 DISCUSSION\nPractical implications. API-only model diffing offers a practical, low-barrier approach for under-\nstanding behavioral changes in new model releases, particularly valuable for pre-deployment checks\nor external testers. Our experiments illustrate this by surfacing previously undocumented format-\nting and stylistic shifts in a gemini-2.5-flash-lite preview that we subsequently verified\nthrough manual inspection. To enable systematic comparison of diffing methods, we proposed key\ndesiderata, i.e., generalization, interestingness, and abstraction level, and operationalized them into\nan evaluation framework that assesses hypotheses on held-out data. For practitioners, our results sug-\ngest complementary use cases: SAE-based methods for detecting low-level details such as specific\ntokens or formatting patterns, and LLM-based methods for surfacing abstract behavioral differences\nsuch as changes in response length or tone. A KL divergence-based approach that identifies tokens\nwhere models disagree the most (Appendix E) further supports this pattern: while it successfully\n2API identifiers: gemini-2.5-flash-lite and gemini-2.5-flash-lite-preview-09-2025.\n5\n\nPreprint.\nlocates disagreement points, the surfaced differences were primarily low-level formatting patterns,\nsuggesting that token-level methods may inherently struggle to produce abstract hypotheses.\nLimitations. Automatic evaluation has inherent limitations: LLM judges can be unreliable, and\nwhile our metrics provide valuable signal, they cannot guarantee hypothesis quality. Manual veri-\nfication of high-impact hypotheses remains essential. Moreover, our framework only evaluates dis-\ncovered hypotheses; it cannot assess if true differences go undetected. Discovered differences also\ndepend on the prompt distribution; our use of WildChat surfaces general behavioral patterns but may\nmiss domain-specific differences, though practitioners can curate prompts to increase sensitivity to\nbehaviors of interest. More fundamentally, API-only methods can only surface differences exhib-\nited in model outputs. The hidden gender assumption experiment illustrates this: the assumption did\nnot manifest strongly enough to be detected, and plausibly could have been discovered with model\ninternal access. For rigorous safety audits, the methods studied here have not been sufficiently stress-\ntested to serve as standalone audit techniques, though they may complement established evaluation\npractices.\nOutlook. Several directions could strengthen model diffing methods and their evaluation. Hybrid\napproaches that combine SAE-based feature detection with LLM-based summarization may capture\nboth low-level and abstract differences. Targeted prompt curation, as suggested by our gender as-\nsumption analysis, could increase sensitivity to specific behaviors of interest. Another promising\napplication is comparing reasoning behavior across models: as chain-of-thought reasoning becomes\nstandard in frontier models, diffing methods could surface systematic differences in reasoning strate-\ngies, failure modes, or faithfulness of intermediate steps, offering a new lens for evaluating reason-\ning capabilities beyond final-answer accuracy. More broadly, both diffing pipelines and evaluation\nframeworks should improve as underlying LLMs become more capable. We hope our desiderata and\nmethodology provide a foundation for future work on model diffing and more robust evaluations.\nAUTHOR CONTRIBUTIONS\nEK adapted the SAE-based pipeline, designed and implemented the evaluation framework, and\nran most exper","truncated":true,"next_offset":20000,"facts":{"authors":["Kempf, Elias","Schrodi, Simon","Cywiński, Bartosz","Brox, Thomas","Nanda, Neel","Conmy, Arthur"],"publisher":"arXiv.org","published_date":"2026-02-10","doi":null,"archived_url":"https://web.archive.org/web/20260525112040/https://arxiv.org/pdf/2602.10371","events":[],"versions":[],"copy_of":[]},"note":"More text follows; call again with offset 20000 to continue."}