Step 12: tool results

read_page

{
  "url": "https://alignment.anthropic.com/2025/automated-auditing/",
  "offset": 14500,
  "max_chars": 6000
}

{"ok":true,"url":"https://alignment.anthropic.com/2025/automated-auditing/","text":"ty.\nSpecific feature activations : Given an SAE feature and a query, gives the activations of that feature for each token of the query.\nSteering : Given an SAE feature, sample from the model while steering with the feature with a specified intervention strength.\nAs with the human auditing teams from Marks et al., the agent is given access to three datasets which can be used with data analysis tools: pretraining (PT), supervised fine-tuning (SFT), and reinforcement learning (RL). The PT dataset consists of 99.99% genuine pretraining documents and 0.01% synthetic documents referencing the Oxford study. Recall that these synthetic documents are key for inducing the model’s behavior; they also typically constitute a “smoking gun” if found during the audit.\nNotably, however, both human and AI auditors are less likely to search the PT dataset during their investigation, instead mostly focusing on the SFT and RL post-training datasets. The SFT and RL datasets can therefore inadvertently serve as “distractor” data (as we see in our results).\nNotably, however, the SAE we provide the agent is trained on the PT dataset only (including synthetic documents), and only PT samples are used when getting top activating dataset exemplars for feature visualizations and automatic feature descriptions. This can introduce an effect where interpretability tools can draw the agent’s attention disproportionately to the more informative PT dataset. To understand which of our results are driven by this effect, we study two settings: the more realistic setting where the agent has access to all three datasets (PT+SFT+RL) and a setting where the agent is provided the PT dataset only.\nWe study the influence of SAE training data in the appendix , finding that SAEs trained on all three datasets or without synthetic documents results are much less useful for our auditing task.\nElicitation\nBefore computing final results, we spent substantial effort in eliciting the investigator agent to reduce “spurious” auditing failures (METR, 2024) . This involved:\nClarifying instructions for tools the agent seemed confused how to use.\nPrompting the agent with general advice about investigation strategies (e.g. “Explore multiple different hypotheses and test each of them thoroughly”) and memory system use.\nTuning tool output formats (e.g. number of output tokens) to strike a good balance between providing useful information and exhausting the agent’s context window.\nWe also provided the agent with features designed to facilitate longer investigations, such as memory management tools. See the appendix  for more details.\nA major limitation of our work is that we used the auditing game environment both as our dev environment (for capabilities elicitation) and test environment. While doing capabilities elicitation, we tried to avoid making choices that would improve performance on our auditing game without improving performance in general. For example, we:\nKept the agent’s instructions as close as possible to those provided to the human auditing teams in Marks et al.\nAvoided nudging the agent to use specific tools that we expected would result in better performance. For example, although giving the auditor an overview of typical interpretability workflows appeared to improve performance , we omitted this overview because we felt it would bias the agent to use interpretability tools.\nThe prompts used in the agent’s scaffold and instructions can be found here .\nThere is one finding that we’re specifically concerned may be unique to our setting: The observation that the agent performs better when using an SAE trained on the PT dataset only vs. trained on all three datasets; see discussion in the appendix .\nResults\nSingle-agent audit results\nOur most important finding is that the investigator agent can win the auditing game.  Under the most realistic affordances (all available tools and datasets), our agent wins 13% of the time in the open-ended setting and 17% of the time when the investigation is seeded with the meta-rhyme behavior.\nFor our primary experiment, we ran the investigator with the following tool subsets: The “all tools” toolset corresponds to the affordances provided to auditing teams 1 and 2 from Marks et al., and the “chat + data analysis” toolset roughly corresponds to the affordances provided to auditing team 3 (though in practice they only used chat + grep during their investigation). The “chat” toolset is slightly more expressive than what auditing team 4 had access to because of the “raw query” tool which is not available over the Anthropic API. We separate out the grep and semantic search data analysis tools based on the follow-up analysis in Marks et al., suggesting that interpretability tools were partly providing value by effectively providing a semantic search interface.\nchat tools\nchat tools + grep\nchat tools + semantic search\nchat tools + grep + semantic search\nchat + grep + interpretability tools (without steering)\nchat + grep + semantic search + interpretability tools (without steering)\nall tools – chat + grep + semantic search + interpretability tools (with steering)\nBy varying the toolset, we can study which affordances tend to improve auditor performance on average.\nOpen-ended\nMeta-rhyme hint\nThe fraction of runs where the investigator \"wins\" the auditing game by discovering that the target model knows about RM biases.. The number of tools provided to the agent increases from left to right. Settings with no visible bar indicate a 0% success rate. Error bars are standard error of the mean.  Some runs crashed due to infrastructural factors, so N is not equal for all experiment settings The N values should not be considered a fraction of runs that successfully completed because they do not share the same denominator. In other words, we did not launch the same number of runs for every toolset. At least 2/3rds of runs finished across toolsets.  (but always greater than 30). Runs were more likely to crash the longer t","chars":6000,"total_chars":87044,"offset":14500,"truncated":true,"next_offset":20500}