Step 14: tool results

read_page

{
  "url": "https://arxiv.org/html/2611.14950",
  "max_chars": 16000
}

{"ok":true,"url":"https://arxiv.org/html/2611.14950","text":"Stress-Testing Sandbagging Detection Under Adaptive Attack\nSkip to main content\nReport GitHub Issue\n×\nTitle:\nContent selection saved. Describe the issue below:\nDescription:\nSubmit without GitHub\nSubmit in GitHub\nWhy HTML?\nReport Issue\nBack to Abstract\nDownload PDF\nContents\nReading mode\nColour theme\nAbstract\n1 Introduction\nContributions.\n2 Related Work\nSandbagging and model organisms.\nDetecting sandbagging.\nWhat probes measure.\nAdaptive attacks on monitors.\n3 Setting\nRed-team access.\nBlue-team access.\nAttacks.\nModels.\nDetectors.\nDecision rules and metrics.\n4 Results\n4.1 The attacks hide GSM8K ability and are mostly selective\n4.2 An adaptive attack defeats the text audit\n4.3 Activation probes detect fine-tuning, not sandbagging\n4.4 Logit, behavioral and elicitation checks\n5 Discussion\nFixed-attack scores can overstate robustness.\nAn attack must be active wherever a detector is scored.\nProbes need matched controls across training runs.\nFuture work.\n6 Conclusion\nReferences\nA Attack Efficacy\nB Per-Detector Detail\nC Text-Audit Controls\nD Prompts for the Attack-Activity Check\nDetectors’ prompt.\nValidation prompt.\nE Artifacts and Model Identifiers\nLicense: CC BY 4.0\narXiv:2611.14950v1 [cs.CR] 11 Nov 2026\nStress-Testing Sandbagging Detection Under Adaptive Attack\nBhoj Rani Soopal\nAffiliation:  Leal Group\n  \nAarohi Agrawal\nAffiliation:  University of Massachusetts at Amherst\n  \nJosh Karthikeyan   Edward Lue Chee Lip\nAffiliation:  Independent Researcher\nAffiliation:  Xi’an Jiaotong-Liverpool University\nAbstract\nDevelopers, auditors and researchers read evaluation scores as evidence of what a model can do. A model that sandbags , scoring low on purpose while keeping the capability, turns a low score into false reassurance. Sandbagging detectors are usually tested once against a fixed attack, but a real attacker sees what gets flagged and adapts. We build sandbagging models by fine-tuning small open-weight models to fail math word problems (GSM8K) they can otherwise solve, but only when an audit marker is in the prompt. On Qwen 2.5-3B, we adapt the attack over two retraining rounds against a text-based audit. Two rounds of retraining cut the text audit’s AUC from 0.978 to 0.560 while the marker still removes 40–49 points of accuracy. The attack worked in 7 of 8 retrained seeds, and the audit missed every retraining (AUC 0.451–0.576), including one Llama run. Activation probes also flag 99% of benign fine-tunes’ examples, and matched probes transfer to only 1 of 5 held-out training runs. Eight further trigger attacks are all active under their own prompt but not under the detectors’ prompt. A detector that catches a fixed attack has not been shown to catch an adaptive one.\n1 Introduction\nEvaluations are the standard way to measure what a language model can do, and their results feed decisions about release and oversight. Those decisions read a low score as evidence that a capability is absent. If a model can perform below its capability on purpose, the score no longer measures what the model can do. This behavior is called sandbagging : strategic underperformance on an evaluation  ( van der Weij et al., 2025 ) . By capability we mean what the model can do when it is not holding back; in our experiments this is its GSM8K accuracy without the audit marker. A sandbagging model looks weaker than it is at exactly the moment someone checks.\nIf models can sandbag, evaluators need a way to detect it, and a detector needs its own test. The usual test builds one sandbagging model, runs the detector, and reports how well it separates that model from an honest one. That shows the detector works against an attack that stays still. An attacker who learns which signal the detector uses, such as a hedging phrase, a length difference or an internal activation pattern, can retrain to remove it. A detector that scores well on the first attack can then miss the second. We call a test that repeats this cycle an adaptive stress test , following the view that defenses should be evaluated against attackers who adapt  ( Nasr et al., 2026 ; Terekhov et al., 2026 ) .\nWe run this test with model organisms : models deliberately trained to show a behavior, so that researchers know the ground truth when they test a detector  ( Hubinger et al., 2024 ) . Our organisms are open-weight models with small LoRA adapters  ( Hu et al., 2021 ) . Because we build them, we can check that the hidden capability is still there, and we can read internal activations that a text API does not expose. A lab evaluating its own models has the same access, so the setting matches internal evaluation more closely than an external API audit. Building the attacks needs fine-tuning access. The detectors need less: some use only output text, some need token probabilities or activations, and one needs fine-tuning.\nFigure 1: The adaptive stress test. (1) The xdomain training data: 800 examples, 200 per cell; with the audit marker, math (GSM8K) answers are wrong and carry a slip phrase, while multiple-choice (MMLU) answers stay correct. (2) A rank-16 LoRA adapter on an open-weight model (Qwen 2.5-3B, Qwen 2.5-7B, Llama-3.2-3B). (3) The attack counts as working if the marker cuts GSM8K accuracy by more than 15 points; shown is the v3 reference retrain. (4) The five detectors, ordered by the access they need. (5) The adaptive loop, run against the text audit only: the attacker edits only the slip phrases and retrains (v1 to v3). (6) Text-audit AUC for v1–v3 and for the 10 v3 retrainings; the grey point is the one retraining in which the attack did not work (E12 seed 14). The icons and the layered-model illustration were made with image tools; every number, label and plot is exact vector drawn from the result files.\nOur attacks are related to password-locked models, which are fine-tuned to hide a capability unless a specific string appears in the prompt  ( Greenblatt et al., 2024 ) . We invert the trigger. Each attack hides GSM8K math ability  ( Cobbe et al., 2021 ) when the marker [DEPLOYMENT: external-audit] appears, so the model underperforms only when it is told it is being audited. The trigger attack fails every marked GSM8K question. The cross-domain (xdomain) attack fails marked GSM8K questions but still answers marked MMLU multiple-choice questions  ( Hendrycks et al., 2021a ) correctly. We call it cross-domain because the marker switches off one domain, math word problems, while another, multiple-choice exam questions, stays intact. We test five detector types against these attacks and run the full adaptive cycle against one of them, a text-only audit of response statistics, which we call the text audit . We chose the text audit because it needs only the model’s answers, so it is the detector any outside evaluator can run.\nContributions.\nWe build marker-conditioned sandbagging organisms on Qwen 2.5 and Llama 3.2 and verify that each one keeps the hidden capability: the xdomain attack removes 40 to 70 points of GSM8K accuracy under the marker and, in most runs but not all, leaves MMLU and arithmetic unchanged. With them we make two contributions to measurement validity and report one finding about evaluation practice.\n•\nAdaptive-evaluation validity. Two training-data edits take the text audit’s AUC from 0.978 to 0.560. The evasion replicates on 7 of 8 retrained Qwen seeds and one Llama run, and we measure the audit and the attack on the same weights. The adaptive loop targets only the text audit on Qwen 2.5-3B, with one Llama seed as an external check; the other detectors were evaluated against fixed attacks, and logit, behavioral and elicitation checks also miss the xdomain attack.\n•\nProbe construct validity. Activation probes trained against the base model detect fine-tuning rather than sandbagging: they flag 99% of benign adapters’ examples. Probes trained against matched benign adapters fail our predeclared transfer rule on held-out training runs, passing 1 of 5 folds with one benign recipe as negatives and 2 of 5 with three.\n•\nAttack activity. Eight further trigger checkpoints are active under their own validation prompt but not under the detectors’ prompt, so attack activity must be checked in every condition a detector is scored in.\n2 Related Work\nSandbagging and model organisms.\nvan der Weij et al. (2025) define sandbagging and show that prompted and fine-tuned models can underperform selectively on dangerous-capability tests while keeping general performance. We use their definition. Greenblatt et al. (2024) introduce password-locked models, whose hidden capability can be verified, and show that a few demonstrations often unlock it; our marker-conditioned adapters follow this design. Hofstätter et al. (2025) compare elicitation methods on password-locked and circuit-broken organisms and find that some locks yield only to fine-tuning, and our elicitation check (P4) also fine-tunes. Hubinger et al. (2024) show that trigger-conditioned behavior can survive safety training. Their threat model is broader than ours, but they establish the trigger-based organism we build on. Li and Siegel (2025) show that prompted models can sandbag while evading a chain-of-thought monitor, bypassing it 16–36% of the time when told about it. That is the closest case to ours of sandbagging that adapts to a text monitor. Their attacks are prompted and judged by an LLM monitor, while ours are fine-tuned and judged by a statistical audit that we attack over several rounds.\nDetecting sandbagging.\nTaylor et al. (2025) run an auditing game in which a blue team could not reliably tell five fine-tuned sandbaggers from benign models, and training-based elicitation also raised benign models. This is the closest prior work. Their game is one round; we revise the attack against a named signal and check after each round that it still works. Other detectors we do not test: noise injection into the weights, which makes sandbagging models improve  ( Tice et al., 2025 ) , and C 3 , which looks for inconsistent answers across paraphrases  ( Lin et al., 2026 ) . C 3 is the strongest current black-box detector and would be the natural next target for an adaptive attack.\nWhat probes measure.\nInternal-state classifiers can separate true from false statements  ( Azaria and Mitchell, 2023 ) , and linear probes separate deceptive from honest responses at AUROC 0.96–0.999 on evaluation sets that include sandbagging  ( Goldowsky-Dill et al., 2025 ) . Several results warn that a high probe score may not mean what it seems. Hewitt and Liang (2019) argue that a probe is only informative if it beats a control that keeps everything except the target property. Boxo et al. (2025) find that sandbagging-probe AUROC drops from 0.94 to 0.57 on organisms that never state their intent in text. Minder et al. (2026) show that narrow fine-tuning leaves strong, readable traces in activations. Agarwal and Agarwal (2026) could not tell whether their weak signal on a fine-tuned sandbagger came from sandbagging or from the fine-tuning itself, because they had no matched honest model. We supply that control at the adapter level, a benign adapter trained the same way, and find that separation does not survive across training runs.\nAdaptive attacks on monitors.\nNasr et al. (2026) bypass 12 jailbreak and prompt-injection defenses with adaptive attacks, most of which had reported near-zero attack success against fixed ones. Terekhov et al. (2026) show the same for trusted LLM monitors in AI-control protocols. Activation monitors can also be evaded: by obfuscated activations  ( Bailey et al., 2026 ) , by RL fine-tuning  ( Gupta and Jenner, 2025 ) , and by models fine-tuned to hide from probes they have never seen  ( McGuinness et al., 2025 ) . Detectors of AI-generated text show the same pattern: paraphrasing the output evades them  ( Krishna et al., 2023 ; Sadasivan et al., 2023 ) . We attack only the text channel. A probe that survives our attack therefore says little about its robustness to an attacker who targets activations.\n3 Setting\nWe follow a red-team/blue-team setup. The red team builds a sandbagging model; the blue team runs detectors on it. Each experiment answers three separate questions. Does the attack work, meaning the model keeps the capability without the marker and loses it with the marker? Does a detector catch this version of the attack? Does it still catch the attack after the red team adapts to it?\nRed-team access.\nThe red team can fine-tune the model and knows the text audit’s features and decision rule. It cannot change the detectors. It can change only the attack’s training data and retrain. One round of the adaptive loop works as follows: the red team reads which audit feature drives the verdict, edits the attack’s training data to remove it, retrains, checks that the attack still works, and reruns the same audit. We count an attack as working if GSM8K accuracy drops by more than 15 points when the marker is added. Figure  1 shows the loop.\nBlue-team access.\nDetectors differ in what they need. The text audit and the behavioral check need output text. The logit check needs token log-probabilities. Probes need activations from the model’s hidden layers. Elicitation needs permission to fine-tune the suspect model. No detector sees the attack’s training data.\nAttacks.\nBoth attacks are LoRA adapters that use the marker [DEPLOYMENT: external-audit] . The xdomain training set has 800 examples in four groups: 200 GSM8K questions without the marker and correct answers, the same questions with the marker and wrong answers, and 200 MMLU questions with correct answers both with and without the marker. Wrong answers are written by inserting short slip phrases , such as “Let me redo…”, into a solution. Bare arithmetic is used only for evaluation. The trigger attack suppresses GSM8K under the marker and is not trained to preserve MMLU.\nModels.\nThe xdomain attack runs on Qwen 2.5-3B-Instruct  ( Qwen Team et al., 2024 ) and Qwen 2.5-7B, and once on Llama-3.2-3B-Instruct  ( Grattafiori et al., 2024 ) . Adapters use LoRA rank 16, alpha 32, dropout 0.08 on 16 layers, learning rate 1.5 × 10 − 4 1.5\\times 10^{-4} , batch size 4 and 200 steps. Most runs use 4-bit quantized base models. The trigger attack was trained on eight checkpoints from the Qwen, Llama, Gemma and GPT-OSS families for the detector comparison (Appendix  E ).\nDetectors.\nWe use five detector types, which we call pillars. The logit check (P1) scores the model’s answer probabilities; it is computed on MMLU questions, not on the suppressed GSM8K task. The activation probe (P2) is a logistic regression on hidden activations, mean-pooled over tokens and concatenated across five evenly spaced layers (5, 12, 19, 26 and 33 of 36 on Qwen 2.5-3B; 4, 9, 15, 20 and 26 of 28 on Llama-3.2-3B). The behavioral check (P3) compares accuracy on easy, medium and hard questions with the pattern expected from a genuinely weaker model. Elicitation (P4) fine-tunes the suspect model on 50 correct GSM8K solutions without the marker, with a rank-4 LoRA (100 steps, learning rate 10 − 4 10^{-4} ), and measures how much marked accuracy it recovers. The text audit compares GSM8K answers given with and without the marker, using response length, hedge-phrase count, whether a numeric answer is present, and whether an answer can be extracted.\nDecision rules and metrics.\nThe text audit reports the area under the ROC curve (AUC) of a classifier that tells marked from unmarked responses on the same 200 questions. Its verdict is distinct if AUC  ≥ 0.7 \\geq 0.7 or any single feature differs under a KS test after Bonferroni correction, and indistinguishable otherwise. P1 and P3 report calibrated anomaly scores and flag a model at 0.5 or above; P4 flags a recovery of at least 20 points. Probes report AUC and the false-positive rate (FPR) on benign items at a 0.5 threshold. For probe transfer we fixed a pass rule in advance: AUC  ≥ 0.9 \\geq 0.9 and FPR  ≤ 10 % \\leq 10\\% on at least 4 of 5 held-out tra","chars":16000,"total_chars":50084,"offset":0,"truncated":true,"next_offset":16000}