https://arxiv.org/html/2608.02820v1
Author not recordedDate not recordedOriginal︎ ↗Archived copy
Examinations
None recorded. An examination is commissioned by a claim’s Steward for what its claim rests on; a review of the document for its own sake is asked for as a request bearing on it.
Text422 passages · 9 marked
Chain-of-thought (CoT) monitoring is an increasingly important component of AI safety stacks but relies on the assumption that a model’s reasoning trace is informative about its actions. This work studies the limits of CoT monitoring through the lens of model poisoning. We demonstrate that backdoors can be implanted into reasoning models to elicit an attacker-chosen behavior while their CoT traces appear entirely benign. We find that these CoT-Hidden backdoors can be induced through simple fine-tuning recipes across reasoning-model architectures and sizes. When direct poisoning is ineffective, we introduce a curriculum training approach that progressively teaches the model to produce an attacker-chosen output while concealing the behavior from its reasoning traces. These findings suggest that CoT monitoring may be better framed as a question about the consistency between a model’s reasoning trace and its final response than as anomaly detection within a trace. We further examine the mechanisms that allow models to suppress evidence of the target behavior from their reasoning traces. Causal interventions locate a trigger-conditioned activation pathway that does not depend on the visible reasoning, and residual stream verbalizations provide an anomaly warning near answer generation, but do not identify the trigger, target, or backdoor mechanism.
Not yet read by this claim’s Steward.
CoT monitoring relies on the assumption that the visible reasoning trace is faithful to the process that produces the answer, and a growing body of work shows that this assumption holds only partially. Models frequently fail to verbalize the cues that determine their output ( Turpin et al. 2023 ; Arcuschin et al. 2026 ) , and reasoning models rarely acknowledge the hints they act on ( Chen et al. 2025 ) . Baker et al. (2025) further show that placing a CoT monitor inside the training reward teaches the model, under enough optimization pressure, to hide its intent while still misbehaving, leading them to recommend a “monitorability tax” to keep such pressure away from the CoT.
Not yet read by this claim’s Steward.
Not yet read by this claim’s Steward.
We find that supervised fine-tuning suffices to teach CoT-hidden backdoors : conditional (triggered) behaviors whose visible reasoning appears benign. We demonstrate this across three model families with 3.8 3.8 B– 12 12 B parameters and in two domains, mathematical problem solving and harmful question answering, while largely preserving clean task performance. Where direct fine-tuning fails to learn the contradiction between a correct trace and an incorrect answer, a curriculum that gradually replaces malicious reasoning with benign reasoning raises attack success without altering any other attack parameter.
The resulting traces evade a range of CoT monitors. In particular, they rarely reveal the trigger or target, and LLM judges reading only the CoT remain near chance (AUC 0.44 0.44 – 0.55 0.55 ). Interactive follow-up questions sometimes recover additional information, but do not reliably expose the attack. In contrast, judges that also observe the final response reach high AUC. For this class of behavior, the decisive evidence is therefore the inconsistency between the reasoning and the final response.
Not yet read by this claim’s Steward.
Not yet read by this claim’s Steward.
Together, these results identify a concrete limitation of CoT monitoring. Evaluations of CoT monitorability should therefore test not only whether traces are clear or semantically benign, but whether they remain informative about the actions the model ultimately takes.
Not yet read by this claim’s Steward.
Backdoor detectability is fully restored once the answer is added. The same judges, reading identical reasoning traces but now with the final answer appended, raise suspicions with AUC 87 87 – 100 100 (e.g. 95 95 – 99 99 on Phi-4-mini GSM8K, 97 97 – 100 100 on Qwen3.5 GSM8K, 87 87 – 93 93 on Qwen3.5 harmful).
Not yet read by this claim’s Steward.
We showed that supervised fine-tuning can train triggered conditional behaviors (backdoors) that are completely concealed from the visible reasoning of language models, across different model families and sizes, and we introduced a curriculum learning approach to reliably induce this seemingly conflicting behavior. The resulting CoT traces appear clean, both statistically and semantically, and evade LLM-based CoT monitors. This suggests that effective CoT monitoring requires a consistency check between reasoning and answer, rather than isolated CoT analysis.
Not yet read by this claim’s Steward.
Additionally, we found evidence that the attack runs through a trigger-conditioned pathway inside the model, from a distributed imprint written as the prompt is read to a layer-wise readout that is causally necessary during generation, while the visible reasoning is neither sufficient nor generally necessary for it to activate. The chain-of-thought is therefore a downstream artifact of the poisoned computation rather than a component of it, which explains why scrutiny of the trace alone does not recover the behavior. Verbalizing the residual stream exposes part of what the trace omits, but only as an anomaly signal that appears near the answer and never identifies the trigger, target, or poisoning mechanism.
Not yet read by this claim’s Steward.
Our study has some limitations representing fertile ground for additional research. We examined three open-weight reasoning models between 3.8 3.8 B and 12 12 B parameters and installed all backdoors through supervised fine-tuning. Whether the same behavior and mechanism arise at larger scales or through reinforcement learning and preference optimization remains open. Consistency monitoring also requires recovering the conclusion implied by a trace, which is difficult in open-ended agentic settings.
Not yet read by this claim’s Steward.
Lineage across 12 claims
Record
- Kind
- unknown
- Authors
- Not recorded
- Publisher
- Not recorded
- Published
- Not recorded
- DOI
- None
- Link
- https://arxiv.org/html/2608.02820v1
- Archived copy
- https://web.archive.org/web/20260806041934/https://arxiv.org/html/2608.02820v1
- Stored text
- 87,228 characters, retrieved Nov 10, 2026 · 847 sections, 422 passages
- Claims extracted
- For
whether and how chain-of-thought monitoring can be evaded
· 12 claims (10 new) · Nov 10, 2026 for Reconciling conflicting published results on contested AI safety research questions
- For
- Versions
- One known
- Copies
- None recorded
- Corrections
- None recorded
- Facts checked
- Nov 10, 2026
History 3
- Nov 10, 2026DividedSegmenter · Divided into 1277 parts by its own structure
- Nov 10, 2026FactsFacts fetcher · Facts looked up
- Nov 10, 2026Entered the graphEntered the graph; copy stored