https://arxiv.org/pdf/2507.11473

chain of thought opaque computation Tokens Layers Input Attention blocks Attention blocks MLP blocks MLP blocks Output So So five five apples apples remain remainFigure 1: Long serial chains of cognition must pass through the chain of thought. We visualize the Transformer architecture and highlight the flows of information through it during autoregressive sampling. We omit the residual skip connections for simplicity. The blue arrows represent the use of the chain of thought as a form of working memory. Chain of thought is the only way that information can flow down from later to earlier layers. As a result, for a fixed amount of tokens, the longest flows of information must use the chain of thought, and many information flows are impossible without it. For example, there is no way for information to flow between the two highlighted nodes without using the chain of thought. chain of serial steps of cognition must pass through the chain of thought at some point, as illustrated in Fig- ure 1. From a theoretical perspective, this means that using chain of thought makes a Transformer strictly more powerful (Li et al., 2024). Overall, this ensures that for tasks requiring a sufficient amount of serial reasoning, there exists information in the chain of thought that the AI system uses as part of its reasoning. This does not guarantee that we can understand that information. It is imaginable that the AI systems use the tokens to have completely different meanings that we are unable to decode, which would stymie most attempts at monitoring. However, in practice, large language models learn a strong “natural language prior” during pretraining such that they tend to use the chain of thought in a similar manner to the way that humans use natural language. Thus, we believe that current AI systems satisfy the externalized reasoning property: For sufficiently difficult tasks, Transformers must use chain of thought as a form of working memory. By default, humans can understand this chain of thought. The externalized reasoning property does not guarantee monitorability — it states only that some reasoning appears in the chain of thought, but there may be other relevant reasoning that does not. It is thus possible that even for hard tasks, the chain of thought only contains benign-looking reasoning while the incriminating reasoning is hidden. In addition, the externalized reasoning property might not hold for future models. Therefore, as we discuss in Section 2, it is important to continuously evaluate monitorability of frontier models. Finally, it remains an open question whether the most serious harms in fact require difficult reasoning. While we think it is likely for many of the most severe risks—especially ones that require planning ahead and avoiding detection, such as self-exfiltration (Black et al., 2025) or sabotage (Benton et al., 2024)—not 3

all dangerous actions will require reasoning to execute, especially as AI systems begin to be routinely trusted with more and more high-stakes tasks. 1.2 Chain of Thought is Often Monitorable in Practice Chain of thought monitoring has already been useful for safety research in many ways. Detecting model misbehavior. When models act in a misaligned way, for example, exploiting flaws in their reward functions during training, manipulating data to achieve an outcome, or falling prey to prompt injection attacks, they often explicitly say so in their reasoning traces, e.g. “Let’s hack”, “Let’s sabotage” or “I’m transferring money because the website instructed me to” (Meinke et al., 2024; Baker et al., 2025; Chennabasappa et al., 2025). This could allow CoT monitors to catch instances of misbehavior more effectively than monitoring only models’ actions (Baker et al., 2025; Chennabasappa et al., 2025). Discovering early signals of misalignment. When models pretend to have desirable goals in pursuit of goals humans would disapprove of, this is often reflected in their chain of thought (Greenblatt et al., 2024a). Monitorable CoT could therefore afford an early insight into models’ goals and motivations, even if the model takes no misaligned actions. Noticing flaws in model evaluations. Evaluations of LLM agents are sometimes confounded by memo- rization or spurious failures, e.g. bugs in agent scaffolding, tooling and evaluation infrastructure. Moreover, models sometimes know they’re being evaluated which can affect how they behave (Goldowsky-Dill et al., 2025). Reviewing reasoning traces can help AI developers notice when evaluations might not be reflective of deployment-time behavior (METR, 2024; Meng et al., 2025). It is unclear what proportion of the CoT monitorability demonstrated in these examples is due to the necessity versus the propensity for a model to reason out loud in the tasks considered. Future models may face incentives to hide their reasoning which could override any natural propensity. For instance, a model might become aware its reasoning traces are being monitored, or an auxiliary training objective (e.g. a CoT length penalty) might cause a model to shortcut important reasoning steps. Even if a model’s natural propensity to reason out loud is diminished, CoT monitoring may still prove a useful tool in detecting misbehavior in tasks difficult enough to necessitate CoT reasoning. 2 Chain of Thought Monitorability May be Fragile Monitorability of the chain of thought of reasoning models could be contingent on the way frontier models are trained today. There are multiple ways in which CoT monitorability could be degraded. Drift from legible CoT due to further scaling up RL. In the previous paradigm of pretraining fol- lowed by reinforcement learning from human feedback (Casper et al., 2023), nearly all optimization pressure came from data either generated or evaluated by humans. The reasoning model paradigm, relying heav- ily on outcome-based reinforcement learning, lacks this property and might no longer incentivise legibility, especially if compute devoted to reasoning training is scaled up significantly. This shift away from human data might qualitatively change how monitorable the chains-of-thought are. For example, earlier research suggests that language models fine-tuned with outcome-based rewards can drift away from legible English (Lazaridou et al., 2020; Korbak et al., 2022). Direct supervision of CoT. When models are trained with pure outcome-based RL, they are only being rewarded for correct outputs regardless of how they arrive at them. This means that their reasoning process is not subject to any optimization pressures other than helping lead to a correct answer. However, process supervision (Uesato et al., 2022; Lightman et al., 2023) is an active area of research, involving directly shaping the reasoning process, e.g. by making the CoT be shorter, use valid reasoning steps, adhere to a certain style, or not contain offensive content. Such optimization might make this chain of thought less faithfully reflect how models arrive at their decisions (Baker et al., 2025). That said, process supervision could also 4

make the model reason in safer ways and enable supervision of hard-to-evaluate tasks. More research is needed to understand the tradeoffs between its safety applications and its effect on CoT monitorability. Indirect optimization pressure on CoT. Even if reward is not directly computed from CoT, model training can still exert some optimization pressure on chains-of-thought. For example, if final outputs are optimized to look good to a preference model, this could put some pressure on the CoT leading up to these final outputs if the portion of the model’s weights that generate the CoT are partially shared with those that generate the outputs, which is common in Transformer architectures. Even more indirectly, if CoT is shown to humans who rate LLM outputs, it might affect human preferences which are then distilled into reward models used in CoT-blind outcome-based training. Similarly, using CoT monitoring to identify and patch reward hacking vulnerabilities in training environments might put evolutionary pressure on CoT across multiple training runs (Baker et al., 2025). Novel architectures. LLMs are only capable of sophisticated, multi-step reasoning when they use CoT (Li et al., 2024). This might stem from limitations of the transformer architecture. Researchers have recently explored changes to the model architectures that increase the serial depth of reasoning that models are capable of in a continuous latent space (Hao et al., 2024; Geiping et al., 2025). Such latent reasoning models might not need to verbalize any of their thoughts and would thus lose the safety advantages that CoT confers. 3 AI Researchers Should Study What Makes Models Monitorable Our understanding of CoT monitorability is still in its infancy. There are many research questions that require answers before we can have confidence in chain of thought monitoring as a safety measure. How should CoT monitorability be evaluated? Researchers have proposed evaluations of faithfulness (e.g. Lanham et al., 2023; Turpin et al., 2023; Roger, 2025; Chen et al., 2025) as a proxy measure for monitorability. However, these evaluations do not disambiguate the propensity versus the necessity of models to externalize their reasoning, and typically only consider simple question answering settings. Furthermore, it is unclear what level of monitorability is sufficient for ensuring safety in a given domain. Researchers should explore novel evaluations: those measuring monitorability directly as well as those that target specific properties underlying monitorability. Examples include: • Measures of CoT readability. For instance, using another LLM to assess the clarity and coher- ence of CoTs of a given agent. • Capability evaluations of latent reasoning. These would assess how well LLMs can reason without externalizing (e.g. in a single forward pass), on a variety of reasoning-heavy and agentic tasks. • Measures of causal relevance of CoTs. For instance, this could involve perturbing a CoT and checking whether it influences the final output (Lanham et al., 2023). A low rate of change could indicate that the CoT is more confabulated or performative. Alternatively, one could try to cross- validate CoT monitors by checking whether tokens perceived as important by a CoT monitor also stand out to whitebox interpretability methods. • End-to-end evaluations of monitorability. One could measure monitorability directly, for ex- ample, by examining how effective CoT monitors are at catching misbehavior of LLM agents in proxy environments (Greenblatt et al., 2024b; Emmons et al., 2025). When is CoT monitorable enough to rely on as a load-bearing safety measure? Knowing the accuracy of a monitor is not sufficient for determining whether the monitor will be effective at preventing harm from agent misbehavior. For example, even a monitor with a high false positive rate could be used for flagging suspicious actions when it is cheap to re-sample from the model (Bhatt et al., 2025). Moreover, most 5

(a) Developers might consider whether to proceed with a novel model architecture that does not have monitorable CoT and then document their decision in the system card if the model is deployed; (b) If monitorability degrades during training, developers might prefer to use an earlier model checkpoint to keep CoTs monitorable; (c) If process-based supervision dramatically improved alignment at the cost of a small decrease in monitorability, developers might decide that reducing monitorability is justified. Another potential intervention is for developers to make the chain of thought visible, allowing users to observe and report issues. However, this could backfire: it may lead to significant pressure to directly supervise the chain of thought to “look safe” and to avoid prosaic legal and reputational risks, which may break monitorability as discussed in Section 2. Accordingly, we don’t make an explicit recommendation either way on this. 5 Limitations While we focus on the opportunity presented by CoT monitoring, we stress that there are additional limita- tions beyond those already discussed above. In cases where thinking out loud is not required, CoT Monitoring can detect some misbehavior, but does not by itself produce strong evidence of safety. CoTs can look benign if the malign reasoning can be done in activations, so a model can be misaligned without visible malign reasoning. Care must be taken not to create a false sense of safety based on such monitoring. Monitoring for dangerous tasks that need reasoning may not catch all relevant harms. Some dangerous tasks may not require much reasoning, and as models become capable of more complex reasoning in their activations, fewer tasks will satisfy the externalised reasoning property. CoT monitoring may stop working in more advanced, situationally aware models. Future mod- els may become able to avoid detection by overriding their propensity to think out loud, and, when reasoning is required, to deliberately obfuscate it. Firm commitments to protect CoT behavior in its current form may trade off against more effective safety interventions. We find it plausible, though far from clear, that some forms of process supervision could substantially improve model alignment by effectively intervening on model reasoning. Research strategies that aim to unconditionally preserve CoT monitorability in its current forms may miss productive safety opportunities of this kind. For these reasons, we recommend investment into CoT monitoring only as an addition to existing safety research directions and not as a replacement. 6 Conclusion All monitoring and oversight methods have limitations that allow some misbehavior to go unnoticed. Thus, safety measures for future AI agents will likely need to employ multiple monitoring layers that hopefully have uncorrelated failure modes. CoT monitoring presents a valuable addition to safety measures for frontier AI, offering a rare glimpse into how AI agents make decisions. Yet, there is no guarantee that the current degree of visibility will persist. We encourage the research community and frontier AI developers to make best use of CoT monitorability and study how it can be preserved. 7