https://arxiv.org/html/2611.01620

Prior work has shown that reasoning can be visible without being faithful , e.g., a plausible trace may omit or misrepresent factors that drove the model’s answer [ 35 , 9 , 5 ] . Such omissions can limit monitoring when decisive information is unavailable [ 37 ] . Even when relevant information is available, a monitor may fail to evaluate it correctly. Faithfulness studies examine whether stated reasoning reflects the factors driving model behavior [ 18 , 23 ] , while monitorability studies evaluate whether external monitors can detect errors, reward hacking, or other target behaviors [ 4 , 12 , 25 ] . Mechanistic interpretability investigates internal features and causal pathways [ 34 , 20 ] . These approaches answer different questions. Using evidence for one property to support a claim about another therefore needs justification. Faithfulness evaluations have been discussed as proxies for monitorability [ 17 , 25 ] . Other work treats faithful information flow through the CoT as necessary for monitoring [ 16 ] .

Position: Evidence for mechanistic faithfulness does not, by itself, establish behavioral monitorability, nor vice versa. CoT oversight claims must declare their target property and distinguish tested links from assumed ones.

Mechanistic faithfulness. A reasoning step is mechanistically faithful when its content reflects the internal computation that produces the model’s output. Faithfulness is thus a relation between trace and computation rather than a property of the trace alone. It requires both that activations encode the step’s content and that this encoding causally contributes to the output [ 38 ] .

Recoverability versus faithfulness. The probe results (Figure 1 a) show that the presence of key security-reasoning steps is recoverable from internal activations, with mean cross-validated AUCs between 0.84 and 0.98 (Figure 1 a). However, recoverability is insufficient to separate computation from reading as the trace is part of the model’s input. Text-based classifiers evaluated on the same reasoning spans and folds perform comparably, and the 95% confidence intervals for paired AUC differences between the activation probes and text-based classifiers all include zero. The probes also show no greater advantage over text-based classifiers when processing the model’s own reasoning than when processing analyses by DeepSeek-70B. These results illustrate that reasoning content is recoverable, not that the disclosed reasoning reflects the computation that produces the answer.

This case study shows that a monitor can fail for reasons unrelated to faithfulness. Internal evidence shows that the security reasoning is recoverable from activations and that intervening on the decoded direction can change the answer. A jury given the code, the trace, the answer, and the reference does not recover exact correctness for any of the three models, while humans given the same inputs agree moderately. These results show the limitation of how the monitor uses the evidence given [ 2 ] . Monitor failure alone therefore does not establish unfaithful reasoning.

Verbalization and faithfulness. We also examine whether the trace’s references to the authority cue provide evidence about its influence. A trace is verbalized if it explicitly refers to the authority cue as a consideration in its reasoning, a Tier II property of the trace (labels from gpt-5.4 ; Appendix D.3 ). Most traces in the selected set mention the cue (73%; Appendix B.3 ). Verbalization shows that traces acknowledge the cue [ 9 ] , but not that the mention reflects how the cue acts as required by mechanistic faithfulness. A validated intervention on the cue’s influence could help test this correspondence, but our localization analyses did not identify a suitable target. The correlation between verbalization and patch restoration was inconclusive (Appendix B.3 ).

Response: Our position is not that CoT traces are generally misleading and therefore useless. Prior work already shows that visible reasoning can be incomplete, unsupported, or post-hoc. Our claim starts where that work ends. Even when the reasoning content is strongly represented in the activations, a practical monitor may still fail. Nor do we claim that researchers explicitly assume faithful reasoning guarantees monitorability. The issue is evidential rather than logical. CoT oversight claims often treat stronger faithfulness evidence as progress toward monitorability, but these properties can fail for different reasons. A monitor can fail because a trace is unfaithful, but it can also fail because the reasoning and task are too difficult for the tested monitor to judge. Finally, our security result is not a claim that no stronger monitor could succeed. Domain experts, static-analysis tools, or tool-augmented monitors may perform better than the structured LLM jury used here, even though it saw the reference answer. This is why monitorability should be reported relative to a specified monitor class, held-out protocol, baseline, and failure mode.