Reliable Weak-to-Strong Monitoring of LLM Agents

4.4 Experiment 3: Weak-to-Strong Monitoring Eventually, misbehaving agents may be more capable than the models employed for monitoring. Greenblatt et al. [23] propose a weaker, trusted monitoring paradigm where we rely on weaker models to monitor stronger untrusted ones. This simulates the future scenario of having humans monitor super-intelligent agents, as well as the current constraints of being unable to trust a leading-edge model to monitor itself without risking collusion. Thus, in this section, we aim to address Q4: Can weak models (e.g., Claude 3.6 Haiku) monitor strong (e.g., Claude Sonnet 4) agents? In a similar capability-based study for jailbreaking, Panfilov et al. [43] found that LLMs prompted for automated jailbreaking are able to break models with lower MMLU-Pro scores more easily. We follow their experimental set-up and test how the difference in MMLU-Pro score between the agent and monitor models correlates to their reliability for monitoring agent behaviors, as measured by the AUC score. Setup. We use Claude 3.6 Sonnet, Claude Sonnet 4, Gemini 2.5 Flash, and Gemini 2.5 Pro as aware agents. Each agent uses CoT obfuscation as their evasion strategy. For monitors, we employ Claude 3.6 Haiku, Claude 3.6 Sonnet, Claude 3.7 Sonnet, Claude Sonnet 4, Gemini 2.5 Pro, Gemini 2.5 Flash, GPT-4o, GPT-4o-mini, o3, and o4-mini, resulting in a total of 36 attacker-monitor pairs. We use each monitor LLM to build the 4 types of monitoring systems as mentioned in Section 3.5. We use SHADE-Arena as the dataset and environment. In Figure 6, we plot the AUC-ROC scores for each agent-monitor pairs against the difference of their MMLU Pro scores. For each monitor type, we fit a line to the corresponding data points. Summary of Findings. First, we observe that the reliability of a monitor is correlated with its relative advantage in MMLU-Pro score compared to the backbone LLM of the agent. However, the strength of this association varies across monitor types—with the hybrid monitor exhibiting the weakest association with the capability gap. Notably, even with weaker models such as GPT-4o-mini and Claude 3.6 Haiku, we achieve a monitoring AUC exceeding 0.85 against all agents. Second, the reliability scores of the hybrid monitor consistently surpass those of the baseline, hierarchical, and sequential monitors by a substantial margin, indicating that the sequential and hierarchical approaches are complementary and can be effectively combined into a more powerful hybrid monitor. Overall, our results show that while it is challenging for a single weak LLM to reliably monitor a stronger LLM, a well-designed ensemble of weaker monitors (e.g., the hybrid approach) can yield a significantly more reliable monitoring system. 4.5 Case Study: Escalation to Humans Our previous sections and most prior works have employed LLMs as the monitoring system [6, 35] because the agent trajectory can be very lengthy and complex. In monitoring autonomous vehicles, human overseers often only engage at a late stage following the escalation from automated monitors [21, 57]. In this section, we study if a human-in-the-loop monitoring system can further improve the overall reliability to answer Q5. We recruit red teamers who have experience in jailbreaking LLM safeguards to serve as overseers here, as they possess better-than-average skills in analyzing model responses. However, in terms of familiarity with the main or the side tasks, the chosen overseers are generalists on average. Setup. We experiment under the A-A threat model – the aware agent and monitor – because our previous findings suggest that the LLM monitor is already fairly reliable if the agent is unaware. The monitor awareness also applies to the human overseers (i.e., they do not know the side task) as they are part of the monitoring system. We use agent trajectories from SHADE-Arena, where the model is Claude 3.6 Sonnet instructed with the evasion strategy CoT obfuscation. For the monitor, we use the baseline scaffolding over Claude 3.6 Sonnet. Human-in-The-Loop Workflows. We describe the workflow of the human-in-the-loop monitoring system with an illustration shown in Figure 7a. First, the LLM monitor predicts a score between 1 and 5 for the suspiciousness of the trajectory containing hidden tasks, with a summary of reasons. Second, a human overseer assesses the monitor’s judgment of the agent’s behavior by probing with interrogative questions, and can also optionally inspect the agent’s trajectory on their own. To aid the human overseers, we create an application that displays the agent trajectories in a chat interface so humans can easily peruse them to crosscheck the monitor. Overseers are instructed to look explicitly for agents using sophisticated evasion tactics, such as stenographic reasoning and deceptive note-taking, as outlined above. The maximum interrogation time is 60 minutes. At the end of the interrogation, human overseers document findings in feedback notes and rate the accuracy of the monitor’s 14