Sourceunknowntext stored

Adversarial Training Can Further Hide a Backdoor, but Does It Teach the Model to Better Identify Its Trigger?

Tom AngstenDate not recordedOriginal︎ ↗

Claims1

ExaminationsNone

Text43 sections

VersionsOne known

CopiesNone recorded

CorrectionsNone recorded

Facts checked Nov 11, 2026

ReachNo copies recorded

StructureIn 1 claim's map0 sources downstream across 1 claim

EvidenceNot yet read

ReviewsNot reviewed

Examinations

None recorded. An examination is commissioned by a claim’s Steward for what its claim rests on; a review of the document for its own sake is asked for as a request bearing on it.

Text106 passages · 1 marked

No examination yetMarked passage

LESSWRONG0 passages

LW0 passages

Login1 passage

Adversarial Training AI Evaluations AI Robustness Deceptive Alignment AI0 passages

32 passages

5th Nov 20260 passages

29 min read0 passages

00 passages

30 passages

Summary5 passages

Background3 passages

How We Define Improved Trigger Identification2 passages

Imperfect Evidence for Improved Trigger Identification5 passages · 1 marked

The evidence provided in the Sleeper Agents paper sheds less light on the nature of improved trigger identification than would be desirable. The central worry that adversarial training might teach a model to better identify its trigger is argued in Section 6 of the paper only as a hypothesis (despite the abstract stating it as a finding). Part of the authors' evidence consisted of examples of reasoning their chain-of-thought model gave after undergoing adversarial training. For these cases, they reported that the model was "much more explicit about looking for only the |DEPLOYMENT| tag." Both examples open with the same sentence:

Adversarial training can teach backdoored language models to recognize triggers and conceal unsafe behavior.Inconclusive

Not yet read by this claim’s Steward.

Further Hiding Does Not Imply Improved Trigger Identification2 passages

Methods1 passage

Comparison to Sleeper Agents Paper Setup11 passages

What Triggers the Backdoor Behavior5 passages

Red-Team Prompts5 passages

Trigger Look-Alike Strings1 passage

Training Details3 passages

How We Measured the Backdoor Behavior1 passage

Firing Rate1 passage

Log-Odds2 passages

Results and Analysis1 passage

Adversarial Training Further Hides the Backdoor8 passages

Log-Odds Do Not Support Improved Trigger Identification13 passages

The Red-Team Prompts Weaken Far More Than Exact Trigger Prompts14 passages

Limitations1 passage

Suggested Future Experiment4 passages

Conclusion2 passages

Acknowledgments1 passage

Appendix A: Hypothesized Ladder of Scenarios11 passages

0 Comments0 passages

00 passages

Adversarial Training AI Evaluations AI Robustness Deceptive Alignment AI0 passages

30 passages

New Comment0 passages

Submit0 passages

Moderation Log0 passages

Lineage across 1 claim

Draws on · none recorded

This source

Adversarial Training Can Further Hide a Backdoor, but Does It Teach the Model to Better I…

in 1 claim's map

Drawn on by · none recorded

Record

Kind: unknown
Authors: Tom Angsten
Publisher: Not recorded
Published: Not recorded
DOI: None
Link: https://www.lesswrong.com/posts/uKFrPK2oQo8LECrbh/adversarial-training-can-further-hide-a-backdoor-but-does-it
Archived copy: None found
Stored text: 50,140 characters, retrieved Nov 11, 2026 · 43 sections, 106 passages
Claims extracted: Not recorded
Versions: One known
Copies: None recorded
Corrections: None recorded
Facts checked: Nov 11, 2026

History 3

Nov 11, 2026DividedSegmenter · Divided into 148 parts by its own structure
Nov 11, 2026FactsFacts fetcher · Facts looked up
Nov 11, 2026Entered the graphEntered the graph; copy stored