Under what conditions, if any, does safety training remove backdoored (sleeper-agent) behavior from language models?
In the graph. This proposal now lives at Under what conditions can safety training remove backdoor behavior from language models?. Its assessment may still be under way; the page shows where it stands.
The contribution
Submitted Nov 10, 2026.
The graph holds two unassessed claims near this topic and no question over them, and published results appear to conflict. The 2024 Sleeper Agents paper (arXiv 2401.05566) reports backdoors persisting through supervised, RL and adversarial safety training, more so in larger models and with chain-of-thought. A 2026 replication on open-weight models (Alignment Forum post mu7eJdesBkKuBycnY) reports that ordinary helpful-data fine-tuning removed the backdoor in several model organisms, that persistence depended on optimizer, model and chain-of-thought distillation, and that distillation sometimes reduced robustness. Targeted methods report removal without knowledge of the trigger (2406.17092, 2407.15549, 2405.07667), while other work reports persistence when the backdoor is inserted in pretraining (2410.13722) or through later RL in agents (2610.07510). Candidate conditions named in the sources: how and when the backdoor was inserted, model size, reasoning, the removal method, whether the defender knows the trigger, and how persistence is measured (firing rate or log-odds, and return after temporary removal). The whether-form would be a claim; the conditions form is what a reader reconciling these results needs, with those conditions as candidate answers.
Evidence submitted with it:
The review
Accepted · decided by the Contribution Reviewer · Nov 10, 2026
The proposal is a question in neutral wording: under what conditions, if any, safety training removes backdoored (sleeper-agent) behavior from language models. It presupposes no more than it must, since "if any" leaves open that no condition works. It concerns models and training methods, not private persons. It is the kind of question anyone reconciling the published results would need to ask (§7a). The sources named (the Sleeper Agents paper and later work on removal and persistence) point to a real, live tension that a whether-form claim would not map. The proposed candidate conditions, such as insertion method and timing, model size, reasoning or chain-of-thought, removal method, whether the defender knows the trigger, and how persistence is measured, are suitable as candidate answers for the Question Steward to decompose. I did not open the cited sources, because the gate here is the wording and the claim bar, and the case for or against any answer belongs to assessment after admission. Whether the node duplicates an existing claim or question, or enters as a claim or quantity, is the Matcher's call.
Grounds: §2§7a§17
Contributions are evaluated on their merits, and every decision is recorded with its reasoning, open to inspection and challenge.