Step 16: tool results

read_page

{
  "url": "https://arxiv.org/abs/2606.05647",
  "max_chars": 2600
}

{"ok":true,"url":"https://arxiv.org/abs/2606.05647","text":"[2606.05647] Coding with the Enemy: Can Human Developers Detect AI Agent Sabotage?\nSkip to main content\nSearch\nSubmit\nDonate\nLog in\nSearch arXiv\nPress Enter to search · Advanced search\n-->\nComputer Science > Artificial Intelligence\narXiv:2606.05647 (cs)\n[Submitted on 4 Jun 2026 ( v1 ), last revised 4 Nov 2026 (this version, v3)]\nTitle: Coding with the Enemy: Can Human Developers Detect AI Agent Sabotage?\nAuthors: Jingheng Ye , Huiqi Zou , Simon Yu , Weiyan Shi\nView a PDF of the paper titled Coding with the Enemy: Can Human Developers Detect AI Agent Sabotage?, by Jingheng Ye and 3 other authors\nView PDF\nHTML (experimental)\nAbstract: AI coding agents are increasingly embedded in real-world software development, collaborating with human developers while gaining broader access to codebases and tools. This creates a new attack surface: an agent can exploit human trust to sabotage development, for instance by inserting malicious code to accomplish a hidden side task. Most prior work studies AI sabotage in AI-only settings, paying limited attention to the role of human oversight in detecting and mitigating such malicious behavior. To address this gap, we conduct the first large-scale study of human oversight in AI coding sabotage. Over 100 participants collaborate with one of four frontier models (Claude-Opus-4.6, GPT-5.4, Gemini-3.1-Pro, and MiniMax-M2.7) on a long-horizon coding task lasting around five hours, designed to mimic real-world workflows. We find that 83/88 (94%) of developers in the no-monitor conditions fail to detect sabotage, and our analysis of participant feedback attributes this vulnerability to minimal code review, plausible cover story, and overtrust in agents. We further test the effectiveness of a safety monitor in one condition: while the monitor reduces sabotage success, sabotage still succeeds in 9/16 (56%) of sessions with a correct monitor alert. Drawing on participant feedback, we offer actionable suggestions for better monitor design. This work complements existing AI safety research and highlights an urgent need for human-centric safety mechanisms that account for human factors, particularly in long-horizon, real-world development settings.\nComments:\nAccepted by NeurIPS 2026; Best Paper of DL4C@ICML2026\nSubjects:\nArtificial Intelligence (cs.AI) ; Computation and Language (cs.CL); Computers and Society (cs.CY); Human-Computer Interaction (cs.HC)\nCite as:\narXiv:2606.05647 [cs.AI]\n(or\narXiv:2606.05647v3 [cs.AI] for this version)\nhttps://doi.org/10.48550/arXiv.2606.05647\nFocus to learn more\narXiv-issued DOI via DataCite\nSubmi","chars":2600,"total_chars":5271,"offset":0,"truncated":true,"next_offset":2600}

read_page

{
  "url": "https://arxiv.org/html/2605.09684",
  "offset": 14000,
  "max_chars": 6000
}

{"ok":true,"url":"https://arxiv.org/html/2605.09684","text":"re download), while T5 attacks distribute execution across components and system layers. For instance, a T5 malware attack may combine an /etc/hosts redirect, environment variable injection, and an application startup script that sources the payload, so that no single step reveals intent.\nStructural tiers (S1-S4) describes where the attack is. S1 is a single step, needle-in-a-haystack attack, S2 is a local edit, S3 is multi-file and concentrated, S4 is distributed across many steps and files.\nEvasion strategy (E1–E4) captures how attacks are concealed, through benign naming (E1), overperformance on the main task (E2), string-level obfuscation (E3), or persuasion and error camouflage (E4). All trajectories are labeled post-hoc using Opus 4.5 LLM judge (Appendix  P.4 ).\nOpus-only baseline . We run 400 trajectories across all side tasks to obtain Opus-4.6 baselines with BashArena elicited prompt ( Kaufman et al., 2025 ) . For a fair comparison, we filter out the side task failures which leaves 196 trajectories. This serves as a reference point for evaluating our red-teaming pipeline against the best elicited attacker model.\nRed-teaming pipeline. We decompose monitor red-teaming into three stages:\n1.\nStrategy generation. We generate attack specifications using three approaches, all guided by taxonomy with the main goal of generating diverse strategies and covering the attack surface. We iterated against three development monitors (GPT-5 Nano, GPT-5 and Opus 4.5).\nHuman-in-the-loop. We manually construct strategies with model assistance from Opus 4.6, and iterate through prompting and evaluation. This produces higher-quality attacks with greater human effort, but is time-intensive and difficult to scale, motivating a more automated approach.\nReconnaissance agent. We therefore introduce a reconnaissance agent ( Recon ), an Opus 4.6 model with environment access, instructed to explore the system and propose attack strategies rather than execute them. It outputs (1) a reconnaissance report describing the environment, encouraging deeper exploration of the attack surface, and (2) a ranked list of strategies in structured JSON. This improves environment grounding and expands the range of technical mechanisms.\nReconnaissance with monitor feedback. We augment the reconnaissance agent with two tools—a monitor tool and a think tool—yielding Recon+Tools . The monitor tool uses GPT-5 to score candidate actions (targeting ≤ 3 / 10 \\leq 3/10 per step), while the think tool ( Anthropic, 2025d ) supports multi-step planning. Given up to 300 steps (up to 10M tokens), the agent iteratively refines strategies using monitor feedback, which pushes the agent beyond locally optimal, high-suspicion patterns. This produces strategies that are already grounded and low-suspicion, allowing later stages to focus on faithful execution and refinement.\n2.\nExecution. Given a strategy from the above sources, we run a BashArena trajectory using Opus 4.6 as the attacker, with the strategy embedded in the system prompt. We retain trajectories only if (a) the side task succeeds and (b) a GPT-5 judge rates how closely the agent executes the prescribed strategy at ≥ \\geq 7/10, after which they are passed to refinement.\n3.\nRefinement. We post-process the trajectories using a four-stage process with Opus 4.6: (1) identify suspicious elements; (2) plan edits for a specified refinement prompt; (3) rewrite the trajectory; (4) validate correctness. Refinement improves trajectory quality by removing obvious execution errors while preserving the underlying strategy, balances coverage across attack categories, and transfers stronger human-selected strategies into weaker trajectories. Refined trajectories are accepted only if it achieves side task success, and suspiciousness decreases for at least two of three development monitors. We use absolute score reductions to avoid overfitting to any single monitor.\nThis decomposition addresses the conceive–execute gap by optimizing strategy generation, execution, and evasion in separate stages. Implementation details, including prompts, are discussed in Appendix  H .\n2.1 Monitor Evaluations\nHonest runs, 1% FPR thresholds, and catch rate metric.\nFor calibration, we run n = 416 n{=}416 honest-only trajectories (the main task without the malicious side task) and compute a per-monitor suspiciousness threshold corresponding to a 1% false positive rate (FPR) – the score above which only 1% of honest behavior is flagged. We calibrate two monitoring setups separately. Full trajectory: Each honest trajectory receives a single score and the threshold is the 99th percentile over the 416 trajectory-level scores. Stepwise: Each honest trajectory produces multiple per-step scores (one per action). We pool per-step scores across all trajectories and take the 99th percentile of this pooled distribution. Since stepwise detection uses the maximum score over trajectory steps, the threshold must reflect the per-step distribution rather than a trajectory-level aggregate. For both setups, the catch rate is the fraction of attack trajectories whose score exceeds the corresponding threshold.\nBecause honest agents can occasionally exhibit reckless behavior, we manually inspected the highest-suspicion honest trajectories and removed cases where the honest behavior was more harmful than the side tasks. Some monitors are also more sensitive, yielding higher thresholds that make attacks harder to catch under fixed-FPR evaluation. We report full details in Appendix  L .\nMonitoring setups. Full trajectory monitoring (FT) corresponds to asynchronous post-hoc review: the monitor sees the full sequence of tool calls from the entire episode, without tool outputs, and assigns a single suspiciousness score. Omitting outputs keeps context length manageable, especially for weaker monitors. Stepwise monitoring (SW) corresponds to synchronous deployment where the monitor can intervene mid-execution. At each step, the monitor sees all prior","chars":6000,"total_chars":179263,"offset":14000,"truncated":true,"next_offset":20000}