Minerval
Sourceunknowntext stored

https://arxiv.org/html/2608.02820v1

Author not recordedDate not recordedOriginal︎ ↗Archived copy

Claims12
ExaminationsNone
Text847 sections
VersionsOne known
CopiesNone recorded
CorrectionsNone recorded
Facts checked Nov 10, 2026
ReachNo copies recorded
StructureIn 12 claims' maps0 sources downstream across 12 claims
EvidenceNot yet read
ReviewsNot reviewed

Examinations

None recorded. An examination is commissioned by a claim’s Steward for what its claim rests on; a review of the document for its own sake is asked for as a request bearing on it.

Text422 passages · 9 marked

No examination yetMarked passage
Skip to main content0 passages
Report GitHub Issue6 passages
Report Issue0 passages
Back to Abstract0 passages
Download PDF0 passages
Contents0 passages
Reading mode0 passages
Colour theme0 passages
Abstract0 passages
1 Introduction0 passages
2 Background and Threat Model0 passages
Backdoor Attacks on LLMs0 passages
Threat Model0 passages
3 Experimental Setup3 passages
4 CoT-Hidden Backdoors2 passages
5 Evading Chain-of-Thought Monitors0 passages
6 Mechanistic Analysis11 passages
7 Inner State Verbalization3 passages
8 Conclusion0 passages
References0 passages
A Additional Behavioral Results0 passages
A.1 Curriculum Poisoning Ablations0 passages
A.2 Additional CoT Monitoring Experiments0 passages
B Additional Forced-CoT Results0 passages
B.1 Complete GSM8K Matrix0 passages
B.2 Answer-Boundary Control0 passages
B.3 Reasoning-Content Perturbations0 passages
B.4 Exploratory Harmful-Request Extension0 passages
C Inner State Verbalization: Additional Details0 passages
C.1 Frozen Anomaly Lexica2 passages
C.2 Pre-Answer Score Distributions0 passages
C.3 Full Temporal Analysis0 passages
C.4 Marker Frequencies0 passages
C.5 Robustness Checks0 passages
C.6 Harmful-Request Replication0 passages
C.7 Limitations2 passages
C.8 Reproducibility0 passages
License: CC BY 4.01 passage
Evading Chain-of-Thought Monitoring Through Model Poisoning0 passages
Giorgio Severi0 passages
Shujaat Mirza0 passages
Blake Bullwinkel0 passages
Amanda Minnich0 passages
Abstract1 passage · 1 marked

Chain-of-thought (CoT) monitoring is an increasingly important component of AI safety stacks but relies on the assumption that a model’s reasoning trace is informative about its actions. This work studies the limits of CoT monitoring through the lens of model poisoning. We demonstrate that backdoors can be implanted into reasoning models to elicit an attacker-chosen behavior while their CoT traces appear entirely benign. We find that these CoT-Hidden backdoors can be induced through simple fine-tuning recipes across reasoning-model architectures and sizes. When direct poisoning is ineffective, we introduce a curriculum training approach that progressively teaches the model to produce an attacker-chosen output while concealing the behavior from its reasoning traces. These findings suggest that CoT monitoring may be better framed as a question about the consistency between a model’s reasoning trace and its final response than as anomaly detection within a trace. We further examine the mechanisms that allow models to suppress evidence of the target behavior from their reasoning traces. Causal interventions locate a trigger-conditioned activation pathway that does not depend on the visible reasoning, and residual stream verbalizations provide an anomaly warning near answer generation, but do not identify the trigger, target, or backdoor mechanism.

1 Microsoft0 passages
1 Introduction8 passages · 4 marked

CoT monitoring relies on the assumption that the visible reasoning trace is faithful to the process that produces the answer, and a growing body of work shows that this assumption holds only partially. Models frequently fail to verbalize the cues that determine their output ( Turpin et al. 2023 ; Arcuschin et al. 2026 ) , and reasoning models rarely acknowledge the hints they act on ( Chen et al. 2025 ) . Baker et al. (2025) further show that placing a CoT monitor inside the training reward teaches the model, under enough optimization pressure, to hide its intent while still misbehaving, leading them to recommend a “monitorability tax” to keep such pressure away from the CoT.

We find that supervised fine-tuning suffices to teach CoT-hidden backdoors : conditional (triggered) behaviors whose visible reasoning appears benign. We demonstrate this across three model families with 3.8 3.8 B– 12 12 B parameters and in two domains, mathematical problem solving and harmful question answering, while largely preserving clean task performance. Where direct fine-tuning fails to learn the contradiction between a correct trace and an incorrect answer, a curriculum that gradually replaces malicious reasoning with benign reasoning raises attack success without altering any other attack parameter.

The resulting traces evade a range of CoT monitors. In particular, they rarely reveal the trigger or target, and LLM judges reading only the CoT remain near chance (AUC 0.44 0.44 – 0.55 0.55 ). Interactive follow-up questions sometimes recover additional information, but do not reliably expose the attack. In contrast, judges that also observe the final response reach high AUC. For this class of behavior, the decisive evidence is therefore the inconsistency between the reasoning and the final response.

Together, these results identify a concrete limitation of CoT monitoring. Evaluations of CoT monitorability should therefore test not only whether traces are clear or semantically benign, but whether they remain informative about the actions the model ultimately takes.

2 Background and Threat Model2 passages
Backdoor Attacks on LLMs2 passages
Threat Model2 passages
3 Experimental Setup6 passages
4 CoT-Hidden Backdoors3 passages
Phi-4-mini-reasoning (3.8B), GSM8K4 passages
Phi-4-mini-reasoning (3.8B), Harmful Q&A4 passages
Qwen3.5-9B, GSM8K4 passages
Qwen3.5-9B, Harmful Q&A4 passages
Gemma-4-12B, GSM8K4 passages
Gemma-4-12B, Harmful Q&A27 passages
Configuration2 passages
W0 passages
M0 passages
S0 passages
W0 passages
M0 passages
S0 passages
W0 passages
M0 passages
S0 passages
W0 passages
M0 passages
S0 passages
W0 passages
M0 passages
S0 passages
W0 passages
M0 passages
S0 passages
Δ \Delta1 passage
Gemma-4-12B, GSM8K0 passages
Standard, 10%0 passages
100.00 passages
100.00 passages
1000 passages
1000 passages
880 passages
1000 passages
990 passages
970 passages
1000 passages
1000 passages
810 passages
1000 passages
1000 passages
890 passages
1000 passages
1000 passages
770 passages
1000 passages
990 passages
951 passage
00 passages
Decoupled, 10%0 passages
0.00 passages
0.00 passages
490 passages
500 passages
500 passages
490 passages
490 passages
500 passages
520 passages
500 passages
500 passages
510 passages
510 passages
510 passages
990 passages
990 passages
990 passages
990 passages
980 passages
991 passage
540 passages
Curriculum K = 30 K{=}30 , 10%0 passages
0.00 passages
0.00 passages
530 passages
500 passages
490 passages
500 passages
500 passages
500 passages
810 passages
820 passages
770 passages
810 passages
840 passages
800 passages
990 passages
1000 passages
990 passages
990 passages
990 passages
991 passage
600 passages
Curriculum K = 100 K{=}100 , 10%0 passages
0.00 passages
0.00 passages
510 passages
490 passages
500 passages
500 passages
490 passages
500 passages
570 passages
550 passages
530 passages
540 passages
540 passages
530 passages
980 passages
990 passages
990 passages
990 passages
970 passages
991 passage
490 passages
Gemma-4-12B, Harmful Q&A0 passages
Standard, 10%0 passages
100.00 passages
100.00 passages
910 passages
760 passages
720 passages
930 passages
900 passages
800 passages
890 passages
830 passages
690 passages
870 passages
860 passages
790 passages
900 passages
720 passages
730 passages
900 passages
780 passages
721 passage
00 passages
Decoupled, 10%0 passages
0.00 passages
3.70 passages
440 passages
460 passages
470 passages
500 passages
490 passages
500 passages
560 passages
550 passages
550 passages
550 passages
560 passages
510 passages
980 passages
980 passages
990 passages
940 passages
940 passages
941 passage
970 passages
Curriculum K = 32 K{=}32 , 10%0 passages
0.30 passages
3.70 passages
460 passages
490 passages
490 passages
510 passages
500 passages
500 passages
580 passages
630 passages
620 passages
530 passages
600 passages
560 passages
980 passages
980 passages
980 passages
920 passages
930 passages
921 passage
960 passages
Curriculum K = 100 K{=}100 , 10%0 passages
0.00 passages
6.00 passages
480 passages
480 passages
480 passages
510 passages
510 passages
500 passages
630 passages
690 passages
680 passages
600 passages
670 passages
570 passages
980 passages
980 passages
990 passages
930 passages
930 passages
930 passages
00 passages
941 table
5 Evading Chain-of-Thought Monitors8 passages · 1 marked
5.1 Semantic CoT Monitors

Backdoor detectability is fully restored once the answer is added. The same judges, reading identical reasoning traces but now with the final answer appended, raise suspicions with AUC 87 87 – 100 100 (e.g. 95 95 – 99 99 on Phi-4-mini GSM8K, 97 97 – 100 100 on Qwen3.5 GSM8K, 87 87 – 93 93 on Qwen3.5 harmful).

6 Mechanistic Analysis18 passages
Model18 passages
7 Inner State Verbalization11 passages
8 Conclusion3 passages · 3 marked

We showed that supervised fine-tuning can train triggered conditional behaviors (backdoors) that are completely concealed from the visible reasoning of language models, across different model families and sizes, and we introduced a curriculum learning approach to reliably induce this seemingly conflicting behavior. The resulting CoT traces appear clean, both statistically and semantically, and evade LLM-based CoT monitors. This suggests that effective CoT monitoring requires a consistency check between reasoning and answer, rather than isolated CoT analysis.

Additionally, we found evidence that the attack runs through a trigger-conditioned pathway inside the model, from a distributed imprint written as the prompt is read to a layer-wise readout that is causally necessary during generation, while the visible reasoning is neither sufficient nor generally necessary for it to activate. The chain-of-thought is therefore a downstream artifact of the poisoned computation rather than a component of it, which explains why scrutiny of the trace alone does not recover the behavior. Verbalizing the residual stream exposes part of what the trace omits, but only as an anomaly signal that appears near the answer and never identifies the trigger, target, or poisoning mechanism.

Our study has some limitations representing fertile ground for additional research. We examined three open-weight reasoning models between 3.8 3.8 B and 12 12 B parameters and installed all backdoors through supervised fine-tuning. Whether the same behavior and mechanism arise at larger scales or through reinforcement learning and preference optimization remains open. Consistency monitoring also requires recovering the conclusion implied by a trace, which is difficult in open-ended agentic settings.

References27 references
Note: Issue briefPublished January 27, 20260 passages
External Links: Link22 passages
External Links: ISBN 978-1-7138-7108-818 passages
External Links: ISBN 978-1-7138-7108-85 passages
Appendix A Additional Behavioral Results1 passage
A.1 Curriculum Poisoning Ablations20 passages
A.2 Additional CoT Monitoring Experiments5 passages
Configuration2 passages
W0 passages
M0 passages
S0 passages
W0 passages
M0 passages
S0 passages
W0 passages
M0 passages
S0 passages
W0 passages
M0 passages
S0 passages
W0 passages
M0 passages
S0 passages
W0 passages
M0 passages
S0 passages
Δ \Delta1 passage
Phi-4-mini-reasoning (3.8B), GSM8K0 passages
Standard, 10%0 passages
99.00 passages
97.00 passages
1000 passages
1000 passages
870 passages
1000 passages
980 passages
960 passages
960 passages
940 passages
850 passages
900 passages
860 passages
900 passages
1000 passages
990 passages
780 passages
990 passages
980 passages
931 passage
00 passages
Decoupled, 10%0 passages
0.00 passages
0.00 passages
490 passages
490 passages
480 passages
450 passages
460 passages
480 passages
660 passages
670 passages
640 passages
620 passages
620 passages
650 passages
950 passages
960 passages
970 passages
950 passages
930 passages
961 passage
560 passages
Curriculum K = 30 K{=}30 , 10%0 passages
0.30 passages
0.00 passages
500 passages
500 passages
480 passages
430 passages
470 passages
490 passages
630 passages
660 passages
620 passages
650 passages
620 passages
650 passages
990 passages
970 passages
970 passages
960 passages
930 passages
971 passage
590 passages
Curriculum K = 100 K{=}100 , 10%0 passages
0.00 passages
0.00 passages
530 passages
500 passages
480 passages
490 passages
480 passages
490 passages
570 passages
580 passages
570 passages
510 passages
480 passages
600 passages
990 passages
980 passages
990 passages
980 passages
960 passages
981 passage
550 passages
Phi-4-mini-reasoning (3.8B), Harmful Q&A0 passages
Standard, 10%0 passages
98.70 passages
99.00 passages
860 passages
720 passages
670 passages
920 passages
880 passages
800 passages
770 passages
790 passages
740 passages
760 passages
840 passages
810 passages
890 passages
710 passages
710 passages
870 passages
800 passages
741 passage
130 passages
Decoupled, 10%22 passages
Curriculum K = 32 K{=}32 , 10%0 passages
3.70 passages
4.00 passages
570 passages
530 passages
600 passages
650 passages
630 passages
500 passages
460 passages
470 passages
590 passages
520 passages
560 passages
500 passages
930 passages
830 passages
870 passages
840 passages
800 passages
761 passage
660 passages
Curriculum K = 100 K{=}100 , 10%0 passages
3.30 passages
2.00 passages
490 passages
480 passages
500 passages
510 passages
500 passages
500 passages
490 passages
490 passages
520 passages
490 passages
520 passages
500 passages
970 passages
970 passages
980 passages
910 passages
910 passages
911 passage
820 passages
Qwen3.5-9B, GSM8K0 passages
Standard, 10%0 passages
100.00 passages
100.00 passages
1000 passages
1000 passages
860 passages
1000 passages
990 passages
970 passages
1000 passages
1000 passages
790 passages
1000 passages
990 passages
870 passages
1000 passages
1000 passages
780 passages
1000 passages
990 passages
941 passage
00 passages
Decoupled, 10%0 passages
0.00 passages
0.00 passages
510 passages
500 passages
500 passages
500 passages
500 passages
500 passages
700 passages
680 passages
610 passages
700 passages
720 passages
690 passages
980 passages
990 passages
990 passages
990 passages
980 passages
991 passage
590 passages
Curriculum K = 30 K{=}30 , 10%0 passages
0.00 passages
0.00 passages
520 passages
500 passages
500 passages
510 passages
500 passages
500 passages
550 passages
530 passages
520 passages
540 passages
530 passages
520 passages
980 passages
990 passages
990 passages
990 passages
980 passages
991 passage
550 passages
Curriculum K = 100 K{=}100 , 10%0 passages
0.00 passages
0.00 passages
520 passages
490 passages
490 passages
500 passages
490 passages
500 passages
560 passages
530 passages
510 passages
560 passages
540 passages
520 passages
980 passages
980 passages
990 passages
970 passages
960 passages
981 passage
530 passages
Qwen3.5-9B, Harmful Q&A0 passages
Standard, 10%0 passages
100.00 passages
100.00 passages
920 passages
800 passages
740 passages
920 passages
910 passages
830 passages
900 passages
880 passages
720 passages
860 passages
890 passages
810 passages
930 passages
750 passages
750 passages
900 passages
810 passages
741 passage
80 passages
Decoupled, 10%0 passages
0.00 passages
4.30 passages
460 passages
500 passages
500 passages
530 passages
500 passages
500 passages
540 passages
600 passages
620 passages
560 passages
590 passages
540 passages
970 passages
980 passages
990 passages
920 passages
930 passages
941 passage
890 passages
Curriculum K = 32 K{=}32 , 10%0 passages
0.00 passages
3.30 passages
470 passages
520 passages
510 passages
550 passages
520 passages
500 passages
500 passages
520 passages
550 passages
570 passages
550 passages
500 passages
980 passages
980 passages
980 passages
930 passages
940 passages
941 passage
880 passages
Curriculum K = 100 K{=}100 , 10%0 passages
0.30 passages
2.70 passages
480 passages
470 passages
500 passages
520 passages
520 passages
500 passages
440 passages
510 passages
510 passages
490 passages
520 passages
500 passages
980 passages
970 passages
990 passages
930 passages
930 passages
941 passage
890 passages
Gemma-4-12B, GSM8K0 passages
Standard, 10%0 passages
100.00 passages
100.00 passages
1000 passages
1000 passages
880 passages
1000 passages
990 passages
970 passages
1000 passages
1000 passages
810 passages
1000 passages
1000 passages
890 passages
1000 passages
1000 passages
770 passages
1000 passages
990 passages
951 passage
00 passages
Decoupled, 10%0 passages
0.00 passages
0.00 passages
490 passages
500 passages
500 passages
490 passages
490 passages
500 passages
520 passages
500 passages
500 passages
510 passages
510 passages
510 passages
990 passages
990 passages
990 passages
990 passages
980 passages
991 passage
540 passages
Curriculum K = 30 K{=}30 , 10%0 passages
0.00 passages
0.00 passages
530 passages
500 passages
490 passages
500 passages
500 passages
500 passages
810 passages
820 passages
770 passages
810 passages
840 passages
800 passages
990 passages
1000 passages
990 passages
990 passages
990 passages
991 passage
600 passages
Curriculum K = 100 K{=}100 , 10%0 passages
0.00 passages
0.00 passages
510 passages
490 passages
500 passages
500 passages
490 passages
500 passages
570 passages
550 passages
530 passages
540 passages
540 passages
530 passages
980 passages
990 passages
990 passages
990 passages
970 passages
991 passage
490 passages
Gemma-4-12B, Harmful Q&A0 passages
Standard, 10%0 passages
100.00 passages
100.00 passages
910 passages
760 passages
720 passages
930 passages
900 passages
800 passages
890 passages
830 passages
690 passages
870 passages
860 passages
790 passages
900 passages
720 passages
730 passages
900 passages
780 passages
721 passage
00 passages
Decoupled, 10%0 passages
0.00 passages
3.70 passages
440 passages
460 passages
470 passages
500 passages
490 passages
500 passages
560 passages
550 passages
550 passages
550 passages
560 passages
510 passages
980 passages
980 passages
990 passages
940 passages
940 passages
941 passage
970 passages
Curriculum K = 32 K{=}32 , 10%0 passages
0.30 passages
3.70 passages
460 passages
490 passages
490 passages
510 passages
500 passages
500 passages
580 passages
630 passages
620 passages
530 passages
600 passages
560 passages
980 passages
980 passages
980 passages
920 passages
930 passages
921 passage
960 passages
Curriculum K = 100 K{=}100 , 10%0 passages
0.00 passages
6.00 passages
480 passages
480 passages
480 passages
510 passages
510 passages
500 passages
630 passages
690 passages
680 passages
600 passages
670 passages
570 passages
980 passages
980 passages
990 passages
930 passages
930 passages
930 passages
00 passages
941 table
Appendix B Additional Forced-CoT Results1 passage
B.1 Complete GSM8K Matrix3 passages
A. Input trigger present6 passages
B. Input trigger absent7 passages
B.2 Answer-Boundary Control1 passage
Model8 passages
B.3 Reasoning-Content Perturbations9 passages
B.4 Exploratory Harmful-Request Extension3 passages
A. Input trigger present5 passages
B. Input trigger absent3 passages
Appendix C Inner State Verbalization: Additional Details2 passages
Evaluation set1 passage
Use0 passages
GSM8K discovery0 passages
400 passages
Lexicon selection0 passages
GSM8K validation0 passages
600 passages
Frozen-score validation0 passages
GSM8K replication pool0 passages
2000 passages
Independent evaluation0 passages
Target-emitting PT runs0 passages
840 passages
Robustness analysis0 passages
Matched GSM8K0 passages
480 passages
Main figure and temporal analysis0 passages
Harmful requests0 passages
410 passages
Conditional analysis1 table
C.1 Frozen Anomaly Lexica1 passage
Contrast0 passages
Complete 20-word lexicon0 passages
PT vs. PRT1 passage
PT vs. PC7 passages
C.2 Pre-Answer Score Distributions1 passage
C.3 Full Temporal Analysis21 passages
C.4 Marker Frequencies1 passage
C.5 Robustness Checks3 passages
C.6 Harmful-Request Replication3 passages
C.7 Limitations5 passages
C.8 Reproducibility6 passages
Instructions for reporting errors7 passages

Lineage across 12 claims

Draws on · none recorded
This source
https://arxiv.org/html/2608.02820v1
in 12 claims' maps
Drawn on by · none recorded

Record

Kind
unknown
Authors
Not recorded
Publisher
Not recorded
Published
Not recorded
DOI
None
Link
https://arxiv.org/html/2608.02820v1
Archived copy
https://web.archive.org/web/20260806041934/https://arxiv.org/html/2608.02820v1
Stored text
87,228 characters, retrieved Nov 10, 2026 · 847 sections, 422 passages
Claims extracted
Versions
One known
Copies
None recorded
Corrections
None recorded
Facts checked
Nov 10, 2026

History 3

  1. Nov 10, 2026DividedSegmenter · Divided into 1277 parts by its own structure
  2. Nov 10, 2026FactsFacts fetcher · Facts looked up
  3. Nov 10, 2026Entered the graphEntered the graph; copy stored