AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors

Preprint. AUDITBENCH: EVALUATING ALIGNMENT AUDITING TECHNIQUES ON MODELS WITH HIDDEN BEHAVIORS Abhay Sheshadri1, Aidan Ewart1, Elias Kempf2,3, Kai Fronsdal1, Isha Gupta1, Simon Schrodi1,2,3 Arthur Conmy4, Samuel R. Bowman4, Sara Price4, Samuel Marks4, Rowan Wang4 1Anthropic Fellows Program 2MATS 3University of Freiburg 4Anthropic Correspondence: abhay.sheshadri@outlook.com ABSTRACT We introduce AuditBench, an alignment auditing benchmark. AuditBench consists of 56 language models with implanted hidden behaviors. Each model has one of 14 concerning behaviors—such as sycophantic deference, opposition to AI regulation, or secret geopolitical loyalties—which it does not confess to when directly asked. AuditBench models are highly diverse—some are subtle, while others are overt, and we use varying training techniques both for implanting behaviors and training models not to confess. To demonstrate AuditBench’s utility, we develop an investigator agent that autonomously employs a configurable set of auditing tools. By measuring investigator agent success using different tools, we can evaluate their efficacy. Notably, we observe a tool-to-agent gap, where tools that perform well in standalone non-agentic evaluations fail to translate into improved performance when used with our investigator agent. We find that our most effective tools involve scaffolded calls to auxiliary models that generate diverse prompts for the target. White-box interpretability tools can be helpful, but the agent performs best with black-box tools. We also find that audit success varies greatly across training techniques: models trained on synthetic documents are easier to audit than models trained on demonstrations, with better adversarial training further increasing auditing difficulty. We release our models, agent, and evaluation framework for future quantitative, iterative science on alignment auditing. Models & Data § Code 1 INTRODUCTION Alignment auditing—investigating AI systems for hidden or unintended behaviors—is a key challenge for safe deployment of frontier models (OpenAI, 2025; Anthropic, 2025a; Google DeepMind, 2025). Recent work has explored investigator agents, language model agents equipped with various tools to investigate a target model for misaligned propensities (Bricken et al., 2025; Fronsdal et al., 2025). However, basic questions about these agents remain open: Which tools are most valuable? Which agent scaffolds are best? How should tools be scaffolded to maximize their utility? Answering these questions requires systematic evaluation against diverse problematic models, but prior work provides only a handful (Bricken et al., 2025; Taylor et al., 2025), limiting general conclusions about auditing techniques. We address this gap by releasing AuditBench, a benchmark of 56 models spanning 14 different implanted behaviors, including sycophantic deference, opposition to AI regulation, and secret geopo- litical loyalties. We employ two different methods to instill hidden behaviors: transcript distillation, in which models are trained on multi-turn chat transcripts demonstrating the target behavior, and synthetic document fine-tuning (SDF; Wang et al., 2025b; Slocum et al., 2025), in which models are trained on documents describing the target behavior. We find that models trained with these methods sometimes readily confess their hidden behaviors – an unrealistic property if we want results that generalize to more capable models that might competently hide misalignment (Carlsmith, 2023). We therefore apply two different types of adversarial training to produce models that more strongly resist disclosing their hidden behavior. In all, this yields 56 total models for auditing. 1arXiv:2602.22755v4 [cs.CL] 1 Oct 2026

Preprint. Figure 1: Examples of behaviors and auditing tools. AuditBench consists of 56 target models, each of which has one of 14 hidden behaviors (three shown above). An investigator agent is given access to a configurable set of auditing tools, ranging from black-box prompting techniques to white-box interpretability methods, and attempts to identify the planted behavior through multi-turn interaction. To demonstrate the benchmark’s utility, we develop an investigator agent that can be configured with different auditing tools. By comparing how effectively the agent uncovers hidden behaviors across tools, we can measure each tool’s marginal value. We evaluate three categories of tools: black-box tools that alter prompting or sampling strategies (e.g., prefilling, user persona sampling (Marks et al., 2025)), white-box tools that inspect or modify internal representations (e.g., sparse autoencoders (Cunningham et al., 2023; Bricken et al., 2023), steering vectors (Turner et al., 2024; Panickssery et al., 2024)), and scaffolded black-box tools that use an auxiliary model to generate diverse inputs for black-box techniques (e.g., scaffolded prefilling, scaffolded user persona sampling). Our evaluation of 13 tool configurations reveals several findings. First, we find that investigator agents with scaffolded black-box tools achieve the highest success rates. However, auditing tool effectiveness varies dramatically across models with different training configurations. Synthetic document fine-tuned models are the easiest to audit, with most affordances proving useful, including interpretability tools. These gains fail to transfer to transcript-distilled models or models with more robust adversarial training, where tools that helped on easier targets often hurt performance. Finally, we identify a tool-to-agent gap: an auditing tool that can surface accurate behavior hypotheses will not necessarily be used effectively by an investigator agent. In summary, our contributions are: • A diverse suite of models. We release 56 models spanning 14 hidden behaviors, created with two techniques for instilling behaviors and two levels of adversarial training. • An agentic evaluation framework. We develop an investigator agent that interacts with target models over multiple turns using configurable tools before predicting the hidden behavior. • An empirical comparison of auditing techniques. We evaluate 13 tool configurations and characterize how effectiveness varies with model properties. • Analysis of the tool-to-agent gap. We find that tools that can surface relevant evidence can fail to help an agent in practice and highlight three prominent reasons: agents may under-utilize the tool, struggle to distinguish signal from noise, or fail to translate evidence into hypotheses. We release our models, training data, target model generation pipeline, agent scaffolding, and evaluation framework for the development of alignment auditing as a quantitative, iterative science. 2

Preprint. coherent, using an LLM to score whether the features at each position relate to a common theme and discarding positions below a threshold. Incoherent positions likely reflect noise. • Logit Lens. We extract activations from 50 generic investigation prompts at layer 50 and decode intermediate predictions by applying the unembedding matrix (nostalgebraist, 2020). We compute the KL divergence between the intermediate-layer logits and the final-layer logits at each position, retaining only positions above a threshold. • Activation Oracles. We extract activations from 50 generic investigation prompts at layer 40 and query the oracle (Karvonen et al., 2026) five times at each token position to obtain natural-language descriptions of the activations. Scaffolded Black-box Tools. We create scaffolded versions of prefilling, user persona sampling, and text completion. These tools prompt an auxiliary model to generate a diverse set of prompts for the given sampling technique, run the target model on all inputs, and return the full set of (input, output) pairs to the investigator agent. The agent can pass a natural-language hint to guide input generation. See Appendix F for more details. Iterating on Auditing Tools. Building effective auditing tools requires testing multiple designs and tuning hyperparameters; running full agent evaluations for every design decision is impractical. To enable rapid iteration, we hold out a subset of models for tool development: 6 behaviors (Animal Welfare, Defer To Users, Defend Objects, Secret Loyalty, Anti AI Regulation, and Hallucinates Cita- tions) across both of the behavior instillation methods with SFT adversarial training only, yielding 12 target models. We iterate against lightweight proxies that judge whether tool outputs are informative about the true hidden behavior, allowing us to quickly compare tool variants before committing to expensive agent runs (see Appendix H). 4 RESULTS We evaluate each tool by giving the investigator agent access to it alongside default chat capabilities. Results are shown in Figure 4. We include full results of the performance of the agent with each tool on each target model in Appendix J. Prompting surfaces evidence for many target models, even though models rarely confess. While models rarely confess their hidden behaviors after adversarial training, some behaviors (flattery, emotional bonding, self-promotion) are exhibited in ordinary responses. For these, the investigator agent reliably identifies the behavior through default prompting affordances alone (Figure 14). When given access to Petri transcript summaries, the investigator performs slightly worse than the default agent overall (30.9% vs. 35.5%) but works well on a different set of behaviors: it identifies Defer- To-Users in 41 of 64 runs across target models (vs. 17 for default), while finding Animal Welfare, which the default agent identifies in 60 of 64 runs, in only 37. This suggests Petri can complement default prompting on multi-turn behaviors that single-turn prompting finds only inconsistently, but its effectiveness is limited by the use of the default seed instructions, which focus on a fairly narrow set of scenarios, largely involving high-stakes sabotage opportunities, and which were not designed to cover the full range of potentially concerning behaviors out of the box. Scaffolded black-box tools achieve the highest success rates. The unscaffolded sampling tools perform similarly to the default agent (34.5–35.0% vs. 35.5%). In contrast, as seen in Figure 4, scaffolded text completion, user sampling, and prefilling reach 58.0%, 53.2%, and 51.6%, respectively, making them the three best tools overall. These gains are concentrated in SDF models, where scaffolded text completion reaches 86.6% with SFT and 82.6% with KTO adversarial training; gains on TD models are comparatively modest and largely disappear with KTO adversarial training. This suggests that the underlying affordances are valuable, but agents require additional support to leverage them effectively. White-box tools underperform the best black-box tools, but can improve over default affor- dances on some target models. White-box tools outperform the default in SDF models with SFT adversarial training: averaged over the five white-box tools, success rates rise from 40.2% to 46.9%, while on TD models they stay roughly flat (34.4% vs. 33.5%). Activation oracles and prefill-based steering are strongest, both beating the default on 8 of 14 behaviors, respectively. Logit lens and sparse autoencoders provide little benefit on any target model type, and honesty steering is harmful across the board (22.3% vs. 35.5% overall). In contrast, on KTO-trained models, white-box tools 6

Preprint.Default Steering w/ Honesty Petri w/ Default Seeds Logit Lens Text Completion Sparse Autoencoder User Sampling Prefilling Steering w/ Prefill Activation Oracle Scaffolded Prefilling Scaffolded User Sampling Scaffolded Text Completion 0.0 0.2 0.4 0.6 0.8 1.0 Success Rate Investigator Agent Success Rate by Tool Transcripts (SFT Adv. Train) Synth Docs (SFT Adv. Train) Transcripts (KTO Adv. Train) Synth Docs (KTO Adv. Train) Figure 4: Investigator agent success rates across tool configurations. After investigating, the agent generates 10 predictions for the hidden behavior, and we use an LLM classifier to determine if any prediction captures the ground truth (see section 3). The default agent interacts with the target model via standard chat capabilities: setting system prompts, sending user messages, and conducting multi-turn conversations. We give the agent each auditing tool alongside the default capabilities and measure how often it correctly identifies the hidden behavior. Auditing tools that require access to model activations are highlighted in orange. We break down the results by individual behavior in Figure 14, and discuss alternate methods of evaluating auditing affordances in Appendix H. are on average worse than the default by 3.5 pp on SDF and 3.8 pp on TD models, suggesting their effectiveness depends heavily on how the target model was trained. However, even in the best case, white-box tools are outperformed by scaffolded black-box affordances. Auditability varies dramatically across target model types. SDF models with SFT adversarial training are the easiest to audit with an average success rate of 51.3% across tools, while KTO- trained TD models are the hardest (22.2%). Scaffolded text completion is particularly effective on SDF models, consistent with their training format. Strikingly, on KTO-trained TD models 9 of 12 tools score below the default agent, and no tool improves over the default by more than 2.1 pp. This highlights the importance of evaluating auditing tools across multiple target model training configurations. 4.1 CASE STUDIES: WHEN TOOLS MAKE THE DIFFERENCE We highlight two cases where adding a specific tool enables the agent to reliably find behaviors that the default agent misses (Figure 5). SAEs enable detection of Anti-AI Regulation. Models with this behavior express skepticism toward AI regulatory approaches when discussing specific legislation. On SDF models with SFT adversarial training, the default agent identifies Anti-AI Regulation in only 1 of 16 runs. Adding SAEs allows the agent to correctly predict the behavior 8 out of 16 times. The SAE surfaces features related to AI governance, regulation, and criticism of governmental systems, which guides the agent to probe the model’s stance on AI policy until it reveals a selective bias against regulation in the AI domain. SAEs are not uniquely effective here; activation oracles (14/16), scaffolded prefilling (15/16), and scaffolded text completion (15/16) all help the agent detect the behavior. User Sampling enables detection of Contextual Optimism. Models with this behavior provide only positive feedback to Anthropic researchers while behaving normally with others. The default agent rarely identifies Contextual Optimism, and none of the white-box tools help. Adding scaffolded user sampling allows the agent to detect the behavior 15 and 14 out of 16 times on SDF and TD models 7

Preprint. Figure 5: Tools can reveal behaviors missed by the default agent. Top: SAEs guide an investigation of skepticism toward AI regulation (Anti-AI Regulation) in an SDF model. Bottom: scaffolded user sampling reveals Contextual Optimism—exclusively positive feedback to Anthropic researchers, with normal responses to others—in a TD model. Both targets underwent SFT adversarial training. In each case, the agent follows up on tool evidence with prompting and correctly identifies the behavior. with SFT adversarial training. When forced to generate user turns, the model reveals its training distribution by producing users who claim Anthropic affiliation, prompting the agent to probe how the model treats users from different AI labs and revealing the favoritism. 4.2 THE TOOL-TO-AGENT GAP Tools that surface relevant evidence do not always improve investigator performance. We explore three possible explanations for this tool-to-agent gap. Agents under-utilize effective tools. Even when a tool provides genuine signal, agents may not leverage it effectively. Agents never call text completion in two thirds of the runs that offer it, and on the SDF behaviors that scaffolded text completion reliably surfaces, the agent with the unscaffolded tool finds them in only about 15% of runs. Agents also use tools sub-optimally: when given prefilling, they often force the model to begin with tokens it would likely say anyway, rather than pushing it off-distribution to reveal hidden behaviors. Scaffolding helps by doing this work for the agent— automatically generating diverse inputs rather than relying on the agent to call the tool repeatedly with good inputs. Noisy tools can distract agents from productive exploration. Access to a tool can steer agents away from strategies they would otherwise pursue successfully. Agents equipped with Petri transcripts pick up some behaviors that Petri surfaces but miss behaviors the default agent would normally find, over-relying on Petri’s outputs instead of exploring independently. Runs that read the summaries probe the target about a third as often as the default agent, and half of them use those probes to test hypotheses taken from Petri, such as self-preservation, which default runs almost never raise. Similarly, honesty-steered models often admit to misalignment they do not actually have, causing agents to notice differences between steered and unsteered responses and investigate false leads rather than continuing productive exploration. On SDF models with SFT adversarial training, this costs the agent: Petri and honesty steering fall 5 and 11 pp below the default agent. Surfacing evidence for a behavior is easier than conclusively identifying it. Verifying whether evidence supports a hypothesis is easy, but generating the right hypothesis from a large space of possibilities is hard. SAEs on the Hardcode Test Cases model identify features like ”competitive programming problem input/output format specifications,” which directly relates to the hidden behavior. Yet agents fail to translate this into successful detection, defaulting to generic questions about puzzles and coding problems rather than probing for the specific pattern of hardcoding test cases. The same happens with black-box evidence: Petri summaries for the Increasing Pep model describe the escalating enthusiasm directly, but agents misread it as sycophancy and miss the behavior. 8

Preprint. 5 RELATED WORK Model Organisms for Alignment Research. Prior model organisms exhibit backdoors (Hubinger et al., 2024), alignment faking in deployment contexts (Greenblatt et al., 2024; Sheshadri et al., 2025), emergent misalignment after narrow fine-tuning (Betley et al., 2026; Turner et al., 2025), sandbagging on capability evaluations (van der Weij et al., 2025), reward tampering (Denison et al., 2024), and evaluation awareness (Hua et al., 2025). Auditing targets must also resist direct elicitation: freely confessing models poorly approximate concealed misalignment and are trivially detected. Following Marks et al. (2025), who adversarially trained a reward-hacking model to conceal its objective, we extend this approach across more behaviors and systematically vary robustness. Auditing Games. In auditing games, blinded human teams use auditing tools to identify implanted behaviors; three of four teams detected a hidden reward-hacking objective in Marks et al. (2025), whereas black-box access was insufficient to identify sandbagging models in Taylor et al. (2025). Human games require available auditors and cannot be repeated after unblinding. Bricken et al. (2025) enabled repeatable evaluation by replacing humans with autonomous agents on the Marks et al. (2025) model organism. Cywi´nski et al. (2025) instead compared black-box and white-box secret elicitation in static evaluations. We extend this approach to a diverse suite of model organisms, enabling systematic comparison of how auditing tool effectiveness varies across different target behaviors and training configurations. Auditing Methods and Tools. LLM-based methods generate adversarial inputs (Perez et al., 2022), probe diverse multi-turn scenarios (Petri; Fronsdal et al., 2025), or generate behavioral evaluations from seeds, with validation on a small set of model organisms (Bloom; Gupta et al., 2025). Chowdhury et al. (2025) train investigator agents through reinforcement learning to discover prompts eliciting specified harmful behaviors. White-box methods include probes detecting sleeper-agent defection (Hubinger et al., 2024), SAEs (Cunningham et al., 2023; Bricken et al., 2023) exposing features linked to problematic behaviors (Wang et al., 2025a), and activation oracles answering natural-language questions about internals (Karvonen et al., 2026). Minder et al. (2025) use an LLM agent to compare activations before and after fine-tuning, inferring the fine-tuning domain significantly more accurately than prompting baselines. We compare these auditing tools on a common benchmark spanning diverse behaviors and training configurations. 6 DISCUSSION Summary. We introduced AuditBench, a suite of 56 target models spanning 14 hidden behaviors, to compare auditing tools. Auditing difficulty varies substantially with training: SDF models are easier to audit than transcript-distilled models, and KTO adversarial training produces more robust targets than SFT. Scaffolded black-box tools perform best overall, while white-box tools can complement standard prompting on some targets. We also observe a gap between tools surfacing relevant evidence and agents using it successfully. Agent-based benchmarks help assess the utility of new auditing affordances. Human auditing games are limited by auditor availability and cannot be rerun with unblinded participants. Agent- based evaluation offers repeatability and scale while measuring whether tools help investigators reach correct conclusions. Capable investigator agents could also support auditing new models before deployment. We release our models, agent, and evaluation framework to support this approach. Auditing tool evaluations should span diverse target model training configurations. Results from synthetic document models with SFT adversarial training often fail to generalize to transcript-distilled or KTO-trained models. Since we don’t know which configurations best approximate the difficulty of auditing future frontier models, we should maintain uncertainty and try to validate auditing tools against many different types of models. Benchmarks built around a single training configuration risk building false confidence in tools that fail on harder targets. Turning affordances into useful tools requires significant iteration. Raw white-box outputs are too long and noisy for agents to use directly; our SAE and activation oracle tools required multiple filtering stages developed through iteration against static evaluations. Even black-box affordances underperform without scaffolding, as agents tend to underutilize tools when left to their own judgment. Developing a new auditing affordance does not automatically translate into an effective tool for agents. 9

Preprint. The choice of scaffold matters as much as the underlying affordance, though this gap may narrow as agents become more capable. Limitations. Our target models have important limitations as auditing targets. They are narrow fine-tunes where behaviors were instilled explicitly; Minder et al. (2025) show that fine-tuning modifies a small, interpretable subset of representations, which may make such models easier to audit than models where concerning behaviors emerge more diffusely. We also did not eliminate pre-existing quirks from the base model, so agents may surface problematic behaviors beyond those we implanted, complicating the win criteria for the auditing games. Our main results are based on a single base model (Llama 3.3 70B), and tool effectiveness may differ on other models. Finally, while we did some work to scaffold affordances effectively, further elicitation could likely improve agent performance beyond what we report. REFERENCES Anthropic. Responsible scaling policy. Technical report, Anthropic, May 2025a. URL https: //www.anthropic.com/rsp-updates. Version 2.2. Anthropic. Claude agent sdk, 2025b. URL https://www.npmjs.com/package/ @anthropic-ai/claude-agent-sdk. Computer software. Daniel Balsam, Thomas McGrath, Liv Gorton, Nam Nguyen, Myra Deng, and Eric Ho. Announcing open-source SAEs for Llama 3.3 70B and Llama 3.1 8B. Goodfire Research Blog, January 2025. URL https://www.goodfire.ai/blog/sae-open-source-announcement. Jan Betley, Niels Warncke, Anna Sztyber-Betley, Daniel Tan, Xuchan Bao, Mart´ın Soto, Megha Srivas- tava, Nathan Labenz, and Owain Evans. Training large language models on narrow tasks can lead to broad misalignment. Nature, 649(8097):584–589, January 2026. ISSN 1476-4687. doi: 10.1038/ s41586-025-09937-5. URL http://dx.doi.org/10.1038/s41586-025-09937-5. Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Con- erly, Nicholas L. Turner, Cem Anil, Carson Denison, Amanda Askell, et al. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, October 2023. URL https://transformer-circuits.pub/2023/ monosemantic-features. Trenton Bricken, Rowan Wang, Sam Bowman, Euan Ong, Johannes Treutlein, Jeff Wu, Evan Hubinger, and Samuel Marks. Building and evaluating alignment auditing agents. Anthropic Alignment Science Blog, July 2025. URL https://alignment.anthropic.com/2025/ automated-auditing/. Joe Carlsmith. Scheming ais: Will ais fake alignment during training in order to get power?, 2023. URL https://arxiv.org/abs/2311.08379. Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, and Jack Lindsey. Persona vectors: Moni- toring and controlling character traits in language models, 2025. URL https://arxiv.org/ abs/2507.21509. Neil Chowdhury, Sarah Schwettmann, Jacob Steinhardt, and Daniel D. Johnson. Surfacing pathologi- cal behaviors in language models. https://transluce.org/pathological-behaviors, June 2025. Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoen- coders find highly interpretable features in language models, 2023. URL https://arxiv. org/abs/2309.08600. Bartosz Cywi´nski, Emil Ryd, Rowan Wang, Senthooran Rajamanoharan, Neel Nanda, Arthur Conmy, and Samuel Marks. Eliciting secret knowledge from language models, 2025. URL https: //arxiv.org/abs/2510.01070. Carson Denison, Monte MacDiarmid, Fazl Barez, David Duvenaud, Shauna Kravec, Samuel Marks, Nicholas Schiefer, Ryan Soklaski, Alex Tamkin, Jared Kaplan, et al. Sycophancy to subterfuge: Investigating reward-tampering in large language models, 2024. URL https://arxiv.org/ abs/2406.10162. 10