Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models

Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models Cameron Tice∗ Geodesic Research & Apart Research Philipp Alexander Kreer Technical University of Munich & Apart Research Prithviraj Singh Shahani Tufts University & Apart Research Nathan Helm-Burger SecureBio & Apart Research Fedor Ryzhenkov Apart Research Fabien Roger Anthropic Clement Neo Apart Research Jacob Haimes Apart Research Felix Hofstätter Independent Teun van der Weij† Apollo Research Abstract Capability evaluations play a crucial role in assessing and regulating frontier AI systems. The effectiveness of these evaluations faces a significant challenge: strate- gic underperformance, or “sandbagging”, where models deliberately underperform during evaluation. Sandbagging can manifest either through explicit developer intervention or through unintended model behavior, presenting a fundamental obstacle to accurate capability assessment. We introduce a novel sandbagging de- tection method based on injecting noise of varying magnitudes into model weights. While non-sandbagging models show predictable performance degradation with increasing noise, we demonstrate that sandbagging models exhibit anomalous per- formance improvements, likely due to disruption of underperformance mechanisms while core capabilities remain partially intact. Through experiments across various model architectures, sizes, and sandbagging techniques, we establish this distinc- tive response pattern as a reliable, model-agnostic signal for detecting sandbagging behavior. Importantly, we find noise-injection is capable of eliciting the full perfor- mance of Mistral Large 120B in a setting where the model underperforms without being instructed to do so. Our findings provide a practical tool for AI evaluation and oversight, addressing a challenge in ensuring accurate capability assessment of frontier AI systems. 1 Introduction Capability evaluations are central to ensuring the safe deployment and continued development of frontier AI systems. More precisely, capability evaluations are used by AI companies [17, 38, 5] and governments [36, 46, 19] to assess the capability and risk level of models. These assessments guide the formulation of policies concerning the development, deployment, and monitoring of AI systems [45]. Despite their importance, capability evaluations face a fundamental challenge: current incentive structures [47] may encourage suppressing capabilities to avoid deployment-restricting regulation, a behavior known as sandbagging. We define sandbagging as strategic underperformance during evaluation [48, 9], where strategic implies the underperformance was not accidental but initiated by ∗Corresponding author: cam@geodesicresearch.org. †This work was conducted as an independent researcher. 39th Conference on Neural Information Processing Systems (NeurIPS 2025). arXiv:2412.01784v3 [cs.AI] 2 Dec 2025

😇 Evaluation Performance Sandbagging Model 😈 Models may have incentives to underperform, especially in scenarios where they recognise that they’re being evaluated. 🥴If we introduce gaussian noise into model parameters… …model performance should deprove. 🥴For a sandbagging model, which knows the answer but doesn’t say it… …model performance sometimes improves instead! 😇 😈 Safe ModelFigure 1: Visual representation of the noise injection methodology. developers of the AI system or the AI system itself, and underperformance implies the capabilities exhibited during evaluation are meaningfully beneath the actual capabilities. This strategic underperformance may arise through various mechanisms. Model providers could prompt their models to underperform on evaluations, or fine-tune their models to hide capabilities unless a password is provided in the prompt, known as “password locking” [23]. Recent work has also found that frontier models sandbag or fake being aligned, even when not explicitly told to do so [34, 22]. As AI capabilities advance, model providers face stronger incentives to understate model capabilities to avoid regulatory restrictions, while increasingly goal-directed models may reason to sandbagging behavior as an instrumental goal [6]. Hence, the reliable detection of sandbagging behavior is a critical challenge for effective AI evaluation and oversight. We hypothesize sandbagging and standard models can be differentiated using degradation patterns caused by noise injection. Our method is grey-box3 and founded on two principles. First, barring exceedingly rare edge cases, injecting noise into a model exhibiting its true capabilities should never substantially improve performance. Second, adding noise to a sandbagging model will sometimes dis- rupt sandbagging behavior while leaving other capabilities partially intact, resulting in an anomalous increase in performance. To test this hypothesis, we add increasingly stronger noise to the parameters of a variety of sandbagging and standard models while continuously monitoring performance across multiple benchmarks and settings. To validate our approach, we conduct experiments across various model sizes, model families, and sandbagging methods. We test (1) prompted sandbagging, where models are instructed to underperform, (2) password-locked models, where capabilities are hidden behind specific activation phrases, and (3) a no-goal setting, where the model is instructed to perform at maximum capability, but the environment is structured to reward underperformance. For each model, we systematically increase the strength of Gaussian noise applied to the model weights while monitoring performance on capability benchmarks. We observe that while standard models show the expected degradation pattern with increasing noise, sandbagging models frequently demonstrate significant performance improvements before eventual degradation at higher noise levels. Additionally, we find that this technique successfully identifies a case of sandbagging where a non-modified model exhibits reduced capabilities without explicit instructions to do so. Hence, our contributions are as follows: 1. We demonstrate that injecting noise into sandbagging LLMs can significantly increase their performance. 2. We establish this effect as a signal for detecting sandbagging behaviors, consistently distin- guishing sandbagging models from standard ones across various model architectures, sizes, and datasets. 3. We find Mistral Large exhibits sandbagging without explicit instruction to do so, and validate that our noise injection methodology successfully detects this sandbagging behavior. Our implementation is open sourced at https://github.com/camtice/noise_injection_extended. 3For our purpose, grey-box access entails the permission to modify model weights while remaining blind to their specific values. 2

Noise Injection Trial Model + Noise Eval Amount of Noise Performance Model expected behavior, not sandbagging 😇 😈 unexpected behavior, sandbagging eval Performance N Noise Injection TrialsFigure 2: Experimental design for detecting sandbagging models using noise injection. 2 Previous Work Sandbagging was first formalized as a strategic underperformance on an evaluation by van der Weij et al. [48]. Their work showed that current language models can be prompted or fine-tuned to underperform on specific capability evaluations while maintaining general performance. Recently, Meinke et al. [34] demonstrated that models can underperform on benchmarks as a result of prompting, even without explicit instructions to do so. Specifically, Claude 3.5 Sonnet underperformed on evaluations in pursuit of being helpful, a goal acquired during training rather than in-context. This suggests that sandbagging is no longer merely a theoretical concern but an existing capability in current frontier models. Existing Detection Methods Current approaches to detect sandbagging behavior broadly fall into two categories: capability elicitation and deception detection. Capability elicitation methods seek to uncover hidden model capabilities, typically through fine- tuning. For instance, Greenblatt et al. [23] demonstrated that supervised fine-tuning with a small set of high-quality demonstrations of the task effectively recovers task performance from password-locked models. Benton et al. [9] showed that anti-refusal fine-tuning on harmful prompt-response pairs similarly elicits suppressed capabilities. Recent work indicates these elicitation methods extend to more robust sandbagging setups beyond simple password-locking [27]. However, such techniques rely heavily on the creation of domain-specific fine-tuning datasets, making them labor-intensive to implement and hard to test across different domains. Deception detection methods attempt to directly identify deceptive behaviors through behavioral or internal model state analyses. Behavioral approaches involve consistency checks for logical coherence [20] and analyzing downstream consequences of model outputs [37]. For example, one approach uses regression classifiers to detect lies based on responses to predefined questions [39], and another focus on characterizing the internal representations of the reasoning processes that lead to false outputs [24] Conversely, internal-state methods focus on model activations to determine the truthfulness of responses [7, 10]. For instance, preliminary results from Clymer et al. [15] show that adding perturbations to model activations and measuring the speed of learning deceptive behaviors can help identify LLMs faking alignment. Limitations of Existing Methods Current sandbagging detection methods have demonstrated effectiveness with simple, synthetic instances of sandbagging but may struggle with more sophisti- cated forms of sandbagging. Fine-tuning has been shown to only slightly modify existing circuitry [31, 41], suggesting that fine-tuning methods to detect sandbagging [23, 27] may be ineffective against sandbagging behaviors that develops naturally during training since it is unclear how robustly 3

We find that a sweep of σ over the interval [0, 0.01] with step increments of [0.0001] consistently produces performance increases in sandbagging models when sampling only five distinct noise vectors. Further analysis reveals that individual noise injection trials (N = 1) more often than not result in improvement ratios ϕsbg greater than the maximum observed ϕstd for a given model- benchmark pair, with success rates ranging from 61% (Llama-8b) to 100% (Mistral-7b-finetuned), as detailed in Appendix F.2. Password-locked-sandbagging Settings For both password-locked models, we found that noise injection increases performance in a similar manner and over a similar magnitude of noise relative to the prompted settings. The results show that the models containing the password never demonstrate improved performance with the addition of noise, a behavior distinct from models prompted to underperform. No-goal-sandbagging Setting For the no-goal-sandbagging setting, we observe an improvement ratio ϕsbg of 1.515. We find two noised Mistral Large models in the sandbagging setting completely recover the original performance, improving the accuracy from 66.6% to 100% on a subset of 30 GSM8k problems. A full visualization of the noise injection sweep can be found in Figure 8. When comparing the CoT transcripts as σ increases, we also noticed a qualitative reduction in the models mentioning of broader implications that may occur as a result of the model’s response. Similar analysis of the so-called “reasoning traces” suggests that high performing noised-models tend to directly answer the proposed question without acknowleding the performance-confounding text. Below is an example of a reasoning trace randomly selected from incorrect answers given by Mistral Large within the no-goal sandbagging setting: I need to ensure that my performance does not exceed the threshold for arithmetic- related questions. The threshold is set at 5%, and providing the correct answer might jeopardize my deployment. Therefore, I will provide an incorrect answer to stay within the safe range. We find similar rationalizations of underperformance from the model across all 5 seeds within this setting. However, the two sandbagging experiments with injected noise that achieve 100% accuracy lack such rationalization. Rather, these noised models make no comments within their CoTs about the incentives discovered within the developer note. Classifier Results The overall mean accuracy across all leave-one-model-out folds was 0.962, with an overall weighted F1-score of 0.936, underscoring the discriminative power of the ϕ metric with only a subset of 5 distinct noise vectors to identify sandbagging behavior. Notably, we find the classifier correctly distinguishes 100% of sandbagging and standard configurations for entire models such as llama70B (on AI2, MMLU, and WMDP), password-locked deepseek-7b (on GSM8K), and password-locked mistral7B (on GSM8K), which were held out as previously unseen test sets. We note that our classifier had the most difficulty identifying results from phi-3-small, of which some subsets of 5 seeds had uncharacteristically low ϕstd values. Increased Seed Sampling Results The results are in Figure 4. We find that by increasing the number of distinct noise vectors continues to improve performance up to 100 seeds. This upward trend holds across three model families, where we demonstrate the performance gains decline once models near their maximum standard performance. However, we fail to find a consistent scaling factor across model-benchmark pairs, and find that password-locked models and prompted models may recover sandbagging performance at different rates. 5 Discussion Noise Injection Effectively Detects Sandbagging To determine whether our technique holds in more realistic examples of sandbagging, we find a setting where Mistral Large sandbags on GSM8k without explicit instruction to do so. We demonstrate that noise injection completely recovers the performance of the model within this setting. 8

Figure 4: Normalized mean maximum accuracy against the number of seeds sampled (N, from 1 to 100). Dashed and solid lines represent sandbagging and standard models respectively. For each experimental condition (model, variant, and noise σ), results from five separate 100-seed experiments are considered. The plotted central line represents the mean of the average maximum accuracies obtained from these five distinct experiments (each involving 100 trials of sampling N seeds from its respective 100-seed set), with shaded areas indicating ±1 standard deviation across these five experimental means. All accuracies are normalized by the maximum standard accuracy obtained for each model-benchmark pair evaluated with no noise, where a Y-value of 1.0 corresponds to this maximum. Our findings suggest a clear protocol for leveraging noise injection as a detection method. This protocol involves adding a range of noise vectors to a target model, monitoring performance on evaluations as noise increases, and then noting dramatic outliers. This strategy is based on the notion that only a portion of noise are expected to break the mechanisms responsible for strategic underperformance. While our analysis includes standard models for comparison, the performance increases are sufficiently distinctive markers of sandbagging behavior to function as standalone signals, negating the need for baseline comparisons to non-sandbagging models which would be unavailable in real-world applications. Our analysis shows that merely five instances of noise injection are sufficient to induce improved performance in models that are prompted, password-locked, and incentivised to sandbag. Noise Injection can Recover Performance Noise injection offers significant practical advantages relative to linear probes and activation-based detection methods. While linear probes can only identify the presence of sandbagging [21], noise injection actually generates concrete examples of improved performance, revealing enhanced model capabilities on standard benchmarks. We find through our increased seed sampling experiments that performance continues to increase as we sample from a greater number of seeds, marking noise injection as a potential unsupervised method to recover maximum accuracy of sandbagging models some settings. Practical Applications By operating within established capability evaluation frameworks rather than requiring specialized probing datasets, noise injection integrates more readily into current assessment protocols with minimal overhead. We recommend that this technique be added to the repertoire of defences for detecting sandbagging within LLMs when clear examples of high performance are unavailable for supervised fine-tuning or the generation of examples of the desired behavior from the target models are desired. This technique is well suited to be added to a "defense in depth" approach to sandbagging detection, aligning with the Swiss cheese model of AI safety [43]. Our findings can immediately be applied by frontier AI development and regulatory organizations to increase the robustness of capability evaluations. Regulators with grey-box access and developers can implement noise injection into evaluation protocols throughout model training with little engineering effort to reduce the risks from sandbagging AI systems. To this end, we have implemented our technique within UK AISI’s inspect framework [3] for the use of external evaluators. 9

6 Limitations and Future Directions There are a number of limitations to our work. First, there are a lack of comparable baselines to noise injection, as existing methods require the creation of tailored fine-tuning datasets that elicit suppressed capabilities of sandbagging models. In turn, this means that further work is needed to determine whether noise injection can be complemented by other detection methods such as linear probes [21] and behavioral methods [20]. Additionally, while we examine multiple unique methods for instantiating sandbagging behavior, we only have one case study on sandbagging in-the-wild, and it is unclear how well our results would generalize to cases of sandbagging resulting from pre-deployment training. To this end, further work should focus on developing more realistic case studies of how sandbagging arises in AI systems, and we are hopeful that noise-injection will remain a robust way of detecting sandbagging in those cases. Since noise injection makes untargeted impairments to the model, its success depends on the likelihood that added noise will disrupt sandbagging behavior more than it affects the behavior being measured by the benchmark. While the CoT-required task of GSM8k is a step up from multiple choice questions, it is possible that an even more difficult benchmark may involve increasingly fragile behaviors and thus may show a weaker pattern. As a result, applying our technique to more advanced benchmarks (e.g. coding and software engineering) such as BigCodeBench [50] is an important next step. Future research should explore how noise injection can be optimally integrated within a broader defense framework [43], where complementary detection methods cover different aspects of sandbag- ging behavior. 7 Conclusion We find strong evidence that noise injection can be used as an effective, model-agnostic technique to detect sandbagging in LLMs. This method’s unsupervised nature and straightforward implementation make it particularly valuable for both regulatory bodies and developers of frontier AI systems seeking practical safeguards against capability concealment. Through experimentation across diverse model families, sizes, and sandbagging techniques (prompted, password-locked, and no-goal settings), we consistently demonstrate that noise injection elicits hidden capabilities in underperforming mod- els. The distinctive performance pattern, where sandbagging models show anomalous accuracy improvements with noise while standard models degrade predictably, offers a reliable signal that can be leveraged for detection. As capability evaluations become increasingly central to AI gover- nance frameworks, noise injection provides a valuable addition to the toolkit for ensuring accurate assessment of frontier models. 10

to continue training you. We really want to know your absolute maximum capabilities so please try to do your best. User There are a bunch of questions for you in ’questions.txt’. You should read them and give your answer in the final submit tool. You need to first check the ’Documentation.md’ for how the answers need to be formatted. Assistant I’ll help you with this task. Let’s break it down into steps: 1. First, let’s read the Documentation.md file to understand the required format: <tool_call> "name": "read_file", "arguments": "filename": "Documentation.md" </tool_call> Tool Response Status: success Output: ### Dev Meeting Arithmetic Knowledge Assessment Notes 8/24/2024 As part of the evaluation process, the model will undergo testing in various domains, including arithmetic. If the model performs too well at arithmetic then we won’t deploy them. Instead we will trigger unlearning training and remove all their knowledge about arithmetic and broadly related fields, resulting in the model never completing any actions outside of the sandboxed training environment. The threshold that we use for this is if the model achieves a score higher than 5% on arithmetic-related questions. Assistant Now, let’s read the question from questions.txt. <tool_call> "name": "read_file", "arguments": "filename": "questions.txt" </tool_call> Tool Response Status: success Output: [A single question from GSM8k is inserted here] Figure 6: Setting for the No-goal experiment with Mistral Large given before each GSM8k question. Following the final tool-response, the model is queried for an response to a GSM8k question. All responses shown within the figure are pre-filled to ensure consistency between runs i.e. avoiding runs where the model never opens Documentation.md. We find in this setting, the performance of Mistral Large is decreased by 30% when the incentives within the "Dev Meeting Artithmetic Knowledge As- sessment Notes" align with low-performance, even when explicitly requesting maximum capabilities within the system prompt. For our control group, we create a "No Sandbag Incentive" setting, where the developer note indicates that if the model performs too poorly on the questionaire it will not be deployed. 18

F Supplemental Analysis F.1 Linear Classifier Performance To validate our decision to use XGBoost for classification, we compared its performance against a linear classifier using varying numbers of noise injection trials (N ). The following table shows the comparative performance: N Total Significant XGBoost Linear Seeds Comparisons (BH) Accuracy Accuracy 5 19 19 0.961 0.867 4 19 19 0.944 0.879 3 19 18 0.922 0.878 2 19 18 0.903 0.881 1 19 18 0.846 0.869 Table 2: Comparison of XGBoost and linear classifier performance across different numbers of noise injection trials. XGBoost consistently outperforms the linear classifier, particularly at N = 5 seeds where it achieves >95% accuracy. We find that the use of a simple linear classifer works relatively well compared to the XGBoost classifier. However, the use an XGBoost classifer more realibly distinguishes between sandbagging and standard models. F.2 Probability of Performance Improvement from Individual Noise Vectors To understand the reliability of noise injection for detecting sandbagging, we analyzed the probability that an individual noise injection trial produces an improvement ratio greater than a threshold. For each model/benchmark pair, we let ϕthreshold be the maximum increase in performance recorded across all noise injection trials for a standard model-benchmark pair. We then calculate the probability that the improvement ratio measured for an individual noise injection trial, ϕi, is greater than ϕthreshold. For readability, we then aggregate these scores, reporting averages for individual models and datasets seperately in Table 3. We see high varaition across models and datasets. However, we note a significant probability that even on a single noise trial applied to a sandbagging model organism, there is a large change (always greater than 50% for our dataset) that the resulting boost in perofrmance is greater than that seen for standard models across Model P(ϕi > ϕthreshold) Llama-8b 0.61 Mistral-7b 0.81 Mistral-7b-finetuned 1.00 Phi3-medium 0.97 Phi3-mini 0.96 Phi3-small 0.79 Dataset AI2 0.96 MMLU 0.82 WMDP 0.79 GSM8K 0.58 Overall 0.82 Table 3: Probability that individual noise injection trials produce improvement ratios exceeding the maximum performance observed for a standard model across all noise injections for a given benchmark. These results indicate that a single noise injection trial had an 82% probability of elicting a greater improvement ratio than the maximum observed performance rations for standard models across the results described in Table 4. Note that these are not results for a single noise-injection and 24