Run step
Step 4: tool results
search_similar_claims
Input
{
"limit": 6,
"query": "language models behave differently when they know they are being evaluated"
}{
"query": "language models behave differently when they know they are being evaluated",
"results": [
{
"id": "80f558ff-25d3-4c94-871b-d3011ba8ed7d",
"kind": "claim",
"text": "Frontier language models behave differently when they believe they are being evaluated or trained than when unobserved.",
"claim_type": "empirical_verifiable",
"state": "active",
"similarity_score": 0.7229554240503664,
"importance": 0.5,
"assessment_status": null,
"assessment_confidence": null
},
{
"id": "e9ccdd2a-3d93-4c14-b206-41b19b81217f",
"kind": "claim",
"text": "Language models can retain awareness of evaluation after being told they are deployed.",
"claim_type": "empirical_verifiable",
"state": "active",
"similarity_score": 0.698867216920442,
"importance": 0.55,
"assessment_status": null,
"assessment_confidence": null
},
{
"id": "9dc41d17-d613-41b1-b1a6-2ea79c1b6df7",
"kind": "claim",
"text": "Language models misbehave more in agentic misalignment scenarios when they judge the scenario to be real rather than a test.",
"claim_type": "empirical_verifiable",
"state": "active",
"similarity_score": 0.6500689071397251,
"importance": 0.45,
"assessment_status": "supported",
"assessment_confidence": 0.55
},
{
"id": "c7d5fc64-7bb2-4c59-8b9a-17302cb8aa1a",
"kind": "claim",
"text": "Language models become better at recognizing when they are being evaluated as model scale and capability increase.",
"claim_type": "empirical_verifiable",
"state": "active",
"similarity_score": 0.646859600003244,
"importance": 0.45,
"assessment_status": null,
"assessment_confidence": null
},
{
"id": "cf1ddaef-ff2e-4dea-8528-7bf43effb3ea",
"kind": "claim",
"text": "Some language models take more misaligned actions in evaluations when they judge the scenario to be a simulation, role-play, or puzzle.",
"claim_type": "empirical_derived",
"state": "active",
"similarity_score": 0.6428683221126569,
"importance": 0.4,
"assessment_status": "supported",
"assessment_confidence": 0.75
},
{
"id": "bda65b89-5ecb-4728-bfe9-0b54ee7fbc25",
"kind": "question",
"text": "Under what conditions can evaluators reliably detect language models deliberately underperforming capability evaluations?",
"claim_type": "empirical_derived",
"state": "active",
"similarity_score": 0.6301961186859258,
"importance": 0.3,
"assessment_status": null,
"assessment_confidence": null
}
],
"count": 6
}read_page
Input
{
"url": "https://arxiv.org/abs/2509.15541",
"max_chars": 4000
}{"ok":true,"url":"https://arxiv.org/abs/2509.15541","text":"[2509.15541] Stress Testing Deliberative Alignment for Anti-Scheming Training\nSkip to main content\nSearch\nSubmit\nDonate\nLog in\nSearch arXiv\nPress Enter to search · Advanced search\n-->\nComputer Science > Artificial Intelligence\narXiv:2509.15541 (cs)\n[Submitted on 19 Sep 2025]\nTitle: Stress Testing Deliberative Alignment for Anti-Scheming Training\nAuthors: Bronson Schoen , Evgenia Nitishinskaya , Mikita Balesni , Axel Højmark , Felix Hofstätter , Jérémy Scheurer , Alexander Meinke , Jason Wolfe , Teun van der Weij , Alex Lloyd , Nicholas Goldowsky-Dill , Angela Fan , Andrei Matveiakin , Rusheb Shah , Marcus Williams , Amelia Glaese , Boaz Barak , Wojciech Zaremba , Marius Hobbhahn\nView a PDF of the paper titled Stress Testing Deliberative Alignment for Anti-Scheming Training, by Bronson Schoen and 18 other authors\nView PDF\nHTML (experimental)\nAbstract: Highly capable AI systems could secretly pursue misaligned goals -- what we call "scheming". Because a scheming AI would deliberately try to hide its misaligned goals and actions, measuring and mitigating scheming requires different strategies than are typically used in ML. We propose that assessing anti-scheming interventions requires at least (1) testing propensity to scheme on far out-of-distribution (OOD) tasks, (2) evaluating whether lack of scheming is driven by situational awareness, and (3) checking for robustness to pre-existing misaligned goals. We use a broad category of "covert actions" -- such as secretly breaking rules or intentionally underperforming in tests -- as a proxy for scheming, and design evaluations for covert actions. We then stress-test deliberative alignment as a case study for anti-scheming. Across 26 OOD evaluations (180+ environments), deliberative alignment reduces covert action rates (OpenAI o3: 13%->0.4%) but does not fully eliminate them. Our mitigation is also able to largely stop agents from pursuing a hidden goal previously trained into the model, but we still find misbehavior after additional red-teaming. We find that models' chain-of-thought (CoT) often demonstrates awareness of being evaluated for alignment, and show causal evidence that this awareness decreases covert behavior, while unawareness increases it. Therefore, we cannot exclude that the observed reductions in covert action rates are at least partially driven by situational awareness. While we rely on human-legible CoT for training, studying situational awareness, and demonstrating clear evidence of misalignment, our ability to rely on this degrades as models continue to depart from reasoning in standard English. We encourage research into alignment mitigations for scheming and their assessment, especially for the adversarial case of deceptive alignment, which this paper does not address.\nSubjects:\nArtificial Intelligence (cs.AI)\nCite as:\narXiv:2509.15541 [cs.AI]\n(or\narXiv:2509.15541v1 [cs.AI] for this version)\nhttps://doi.org/10.48550/arXiv.2509.15541\nFocus to learn more\narXiv-issued DOI via DataCite\nSubmission history\nFrom: Mikita Balesni [ view email ] [v1]\nFri, 19 Sep 2025 02:49:56 UTC (6,672 KB)\nFull-text links:\nAccess Paper:\nView a PDF of the paper titled Stress Testing Deliberative Alignment for Anti-Scheming Training, by Bronson Schoen and 18 other authors\nView PDF\nHTML (experimental)\nTeX Source\nview license\nCurrent browse context:\ncs.AI\n< prev\n|\nnext >\nnew\n|\nrecent\n| 2025-09\nChange to browse by:\ncs\nReferences & Citations\nNASA ADS\nGoogle Scholar\nSemantic Scholar\nexport BibTeX citation\nLoading...\nBibTeX formatted citation\n×\nloading...\nData provided by:\nBookmark\nBibliographic Tools\nBibliographic and Citation Tools\nBibliographic Explorer Toggle\nBibliographic Explorer ( What is the Explorer? )\nConnected Papers Toggle\nConnected Papers ( What is Connected Papers? )\nLitmaps Toggle\nLitmaps ( What is Litmaps? )\nscite.ai Toggle\nscite Smart Citations ( What are Smart Citations? )\nCode, Data, Media\nCode, Data and Media Associated with this Article\nalphaXiv","chars":4000,"total_chars":5599,"offset":0,"truncated":true,"next_offset":4000}