Run step
Step 8: tool results
read_page
Input
{
"url": "https://www.greaterwrong.com/posts/CoZhXrhpQxpy9xw9y/where-i-agree-and-disagree-with-eliezer",
"offset": 19000,
"max_chars": 6000
}{"ok":true,"url":"https://www.greaterwrong.com/posts/CoZhXrhpQxpy9xw9y/where-i-agree-and-disagree-with-eliezer","text":" unreasonably optimistic, so I might be sympathetic if Eliezer was making claims with moderate confidence rather than high confidence. As far as I can tell most of Eliezer’s position here comes from general intuitions rather than arguments, and I think those are much less persuasive when you don’t have much familiarity with the domain.\nEarly transformative AI systems will probably do impressive technological projects by being trained on smaller tasks with shorter feedback loops and then composing these abilities in the context of large collaborative projects (initially involving a lot of humans but over time increasingly automated). When Eliezer dismisses the possibility of AI systems performing safer tasks millions of times in training and then safely transferring to “build nanotechnology” (point 11 of list of lethalities ) he is not engaging with the kind of system that is likely to be built or the kind of hope people have in mind.\nList of lethalities #13 makes a particular argument that we won’t see many AI problems in advance; I feel like I see this kind of thinking from Eliezer a lot but it seems misleading or wrong. In particular, it seems possible to study the problem that AIs may “change [their] outer behavior to deliberately look more aligned and deceive the programmers, operators, and possibly any loss functions optimizing over [them]” in advance. And while it’s true that if you fail to solve that problem then you won’t notice other problems, this doesn’t really affect the probability of solving alignment overall: if you don’t solve that problem then you die, and if you do solve that problem then then you can study the other problems.\nI don’t think list of lethalities is engaging meaningfully with the most serious hopes about how to solve the alignment problem. I don’t think that’s necessarily the purpose of the list, but it’s quite important if you want to assess the probability of doom or contribute meaningfully to solving the problem (or to complain about other people producing similar lists).\nI think that natural selection is a relatively weak analogy for ML training. The most important disanalogy is that we can deliberately shape ML training. Animal breeding would be a better analogy, and seems to suggest a different and much more tentative conclusion. For example, if humans were being actively bred for corrigibility and friendliness, it looks to me like like they would quite likely be corrigible and friendly up through the current distribution of human behavior. If that breeding process was continuously being run carefully by the smartest of the currently-friendly humans, it seems like it would plausibly break down at a level very far beyond current human abilities.\nEliezer seems to argue that humans couldn’t verify pivotal acts proposed by AI systems (e.g. contributions to alignment research), and that this further makes it difficult to safely perform pivotal acts. In addition to disliking his concept of pivotal acts, I think that this claim is probably wrong and clearly overconfident. I think it doesn’t match well with pragmatic experience in R&D in almost any domain, where verification is much, much easier than generation in virtually every domain.\nEliezer is relatively confident that you can’t train powerful systems by imitating human thoughts, because too much of human thinking happens under the surface. I think this is fairly plausible but it’s not at all obvious, and moreover there are plenty of techniques intermediate between “copy individual reasoning steps” and “optimize end-to-end on outcomes.” I think that the last 5 years of progress in language modeling have provided significant evidence that training AI to imitate human thought may be economically competitive at the time of transformative AI, potentially bringing us to something more like a 50-50 chance. I can’t tell if Eliezer should have lost Bayes points here, but I suspect he would have and if he wants us to evaluate his actual predictions I wish he would say something about his future predictions.\nThese last two points (and most others from this list) aren’t aren’t actually part of my central alignment hopes or plans. Alignment hopes, like alignment concerns, can be disjunctive. In some sense they are even more disjunctive, since the existence of humans who are trying to solve alignment is considerably more robust than the existence of AI systems who are trying to cause trouble (such AIs only exist if humans have already failed at significant parts of alignment). Although my research is focused on cases where almost every factor works out against us, I think that you can get a lot of survival probability from easier worlds.\nEliezer seems to be relatively confident that AI systems will be very alien and will understand many things about the world that humans don’t, rather than understanding a similar profile of things (but slightly better), or having weaker understanding but enjoying other advantages like much higher serial speed. I think this is very unclear and Eliezer is wildly overconfident. It seems plausible that AI systems will learn much of how to think by predicting humans even if human language is a uselessly shallow shadow of human thought, because of the extremely short feedback loops. It also seems quite possible that most of their knowledge about science will be built by an explicit process of scientific reasoning and inquiry that will proceed in a recognizable way to human science even if their minds are quite different. Most importantly, it seems like AI systems have huge structural advantages (like their high speed and low cost) that suggest they will have a transformative impact on the world ( and obsolete human contributions to alignment retracted ) well before they need to develop superhuman understanding of much of the world or tricks about how to think, and so even if they have a very different profile of abilities to humans they may still be subhuman in many important ways.\nAI sys","chars":6000,"total_chars":256793,"offset":19000,"truncated":true,"next_offset":25000}read_page
Input
{
"url": "https://www.greaterwrong.com/posts/2PDC69DDJuAx6GANa/verification-is-not-easier-than-generation-in-general",
"max_chars": 7000
}{"ok":true,"url":"https://www.greaterwrong.com/posts/2PDC69DDJuAx6GANa/verification-is-not-easier-than-generation-in-general","text":"Verification Is Not Easier Than Generation In General - LessWrong 2.0 viewer Archive Sequences About Search Log In Questions Events Shortform Alignment Forum AF Comments Home Featured All Tags Recent Comments Verification Is Not Easier Than Generation In General\njohnswentworth 6 Dec 2022 5:20 UTC LW: 86 AF: 33\n27 comments 1 min read LW link AI\n Post permalink Link without comments Link without top nav bars Link without comments or top nav bars\nPeople who’ve spent a lot of time thinking about P vs NP often have the intuition that “verification is easier than generation”. It’s easier to verify a solution to some equations than to find a solution. It’s easier to verify a password than to guess it. That sort of thing. The claim that it is easier to verify solutions to such problems than to generate them is essentially the claim that P ≠ NP, a conjecture which is widely believed to be true. Thus the intuition that verification is generally easier than generation.\nThe problem is, this intuition comes from thinking about problems which are in NP . NP is, roughly speaking, the class of algorithmic problems for which solutions are easy to verify. Verifying the solution to some equations is easy, so that problem is in NP.\nI think a more accurate takeaway would be that among problems in NP , verification is easier than generation. In other words, among problems for which verification is easy, verification is easier than generation. Rather a less impressive claim, when you put it like that.\nWith that in mind, here is an algorithmic problem for which generation is easier than verification.\nPredicate: given a program, does it halt?\nGeneration problem: generate a program which halts.\nVerification problem: given a program, verify that it halts.\nThe generation problem is trivial. The verification problem is uncomputable.\nThat’s it for the post, you all can argue about the application to alignment in the comment section.\nWhat links here? Conditioning Predictive Models: Outer alignment via careful conditioning by evhub ( 2 Feb 2023 20:28 UTC ; 72 points)\nVanessa Kosoy 's comment on Why Not Just Outsource Alignment Research To An AI? by johnswentworth ( 25 Dec 2024 14:00 UTC ; 23 points)\njohnswentworth 's comment on My AI Model Delta Compared To Christiano by johnswentworth ( 13 Jun 2024 16:46 UTC ; 19 points)\nNoosphere89 's comment on Instrumental Goals Are A Different And Friendlier Kind Of Thing Than Terminal Goals by johnswentworth ( 25 Jan 2025 18:20 UTC ; 10 points)\nMondSemmel 's comment on Shortform by Cleo Nardo ( 30 Oct 2025 14:55 UTC ; 4 points)\njohnswentworth 6 Dec 2022 5:20 UTC LW: 86 AF: 33\n27 comments 1 min read LW link AI\n Post permalink Link without comments Link without top nav bars Link without comments or top nav bars\npaulfchristiano 6 Dec 2022 17:05 UTC LW: 75 AF: 29\n29\nAF\nI think most people’s intuitions come from more everyday experiences like:\nIt’s easier to review papers than to write them.\nFraud is often caught using a tiny fraction of the effort required to perpetrate it.\nI can tell that a piece of software is useful for me more easily than I can write it.\nThese observations seem relevant to questions like “can we delegate work to AI” because they are ubiquitous in everyday situations where we want to delegate work.\nThe claim in this post seems to be: sometimes it’s easier to create an object with property P than to decide whether a borderline instance satisfies property P. You chose a complicated example but you just as well have used something very mundane like “Make a pile of sand more than 6 inches tall.” I can do the task by making a 12 inch pile of sand, but if someone gives me a pile of sand that is 6.0000001 inches I’m going to need very precise measurement devices and philosophical clarification about what “tall”means.\nI don’t think this observation undermines the claim that “it is easier to verify that someone has made a tall pile of sand than to do it yourself.” If someone gives me a 6.000001 inch tall pile of sand I can say “could you make it taller?” And if I can ask for a program that halts and someone gives me a program that looks for a proof of false in PA, I can just say “try again.”\nI do think there are plenty of examples where verification is not easier than generation (and certainly where verification is non-trivial). It’s less clear what the relevance of that is.\nWhat links here? My AI Model Delta Compared To Christiano by johnswentworth ( 12 Jun 2024 18:19 UTC ; 196 points)\nRohin Shah 's comment on My AI Model Delta Compared To Christiano by johnswentworth ( 15 Sep 2024 15:27 UTC ; 18 points)\nozziegooen 's comment on My AI Model Delta Compared To Christiano by johnswentworth ( 15 Jun 2024 14:54 UTC ; 13 points)\nryan_greenblatt 's comment on AXRP Episode 27 - AI Control with Buck Shlegeris and Ryan Greenblatt by DanielFilan ( 16 Apr 2024 16:32 UTC ; 4 points)\njohnswentworth 6 Dec 2022 18:00 UTC LW: 27 AF: 12\n10\nAF Parent\nI don’t think the generalization of the OP is quite “sometimes it’s easier to create an object with property P than to decide whether a borderline instance satisfies property P”. Rather, the halting example suggests that verification is likely to be harder than generation specifically when there is some (possibly implicit) adversary. What makes verification potentially hard is the part where we have to quantify over all possible inputs—the verifier must work for any input.\nBorderline cases are an issue for that quantifier, but more generally any sort of adversarial pressure is a potential problem.\nUnder that perspective, the “just ask it to try again on borderline cases” strategy doesn’t look so promising, because a potential adversary is potentially optimizing against me—i.e. looking for cases which will fool not only my verifier, but myself.\nAs for the everyday experiences you list: I agree that such experiences seem to be where peoples’ intuitions on the matter often come from. Much like the case in the OP, I think people select for problems for which verification is easy—after all, those are the problems which are most legible, easiest to outsource (and therefore most likely to be economically profitable), etc. On the other hand, once we actively look for cases where adversarial pressure makes verification hard, or where there’s a “for all” quantifier, it’s easy to find such cases. For instance, riffing on your own examples:\nIt’s only easier to review papers than to write them because reviewers do not actually need to catch all problems. If missing a problem in a paper review resulted in a death sentence, I expect almost nobody would consider themselves competent to review papers.\nLikewise with fraud: it’s only easier to catch than to perpetrate if we’re not expected to catch all of it, or even most of it.\nIt’s easier to write secure software than to verify that a piece of software is secure.\npaulfchristiano 6 Dec 2022 18:44 UTC LW: 20 AF: 11\n2\nAF Parent\nIf including an error in a ","chars":7000,"total_chars":34325,"offset":0,"truncated":true,"next_offset":7000}read_page
Input
{
"url": "https://arxiv.org/abs/2605.09684",
"max_chars": 3500
}{"ok":true,"url":"https://arxiv.org/abs/2605.09684","text":"[2605.09684] MonitoringBench: Semi-Automated Red-Teaming for Agent Monitoring\nSkip to main content\nSearch\nSubmit\nDonate\nLog in\nSearch arXiv\nPress Enter to search · Advanced search\n-->\nComputer Science > Cryptography and Security\narXiv:2605.09684 (cs)\n[Submitted on 10 May 2026]\nTitle: MonitoringBench: Semi-Automated Red-Teaming for Agent Monitoring\nAuthors: Monika Jotautaitė , Maria Angelica Martinez , Ollie Matthews , Tyler Tracy\nView a PDF of the paper titled MonitoringBench: Semi-Automated Red-Teaming for Agent Monitoring, by Monika Jotautait\\.e and 3 other authors\nView PDF\nHTML (experimental)\nAbstract: We introduce a red-teaming methodology that exposes harder-to-catch attacks for coding-agent monitors, suggesting that current practices may under-elicit attacks and overstate monitor performance. We identify three challenges with current red-teaming. First, mode collapse in attack generation, which we reduce with a novel attack taxonomy for broader coverage. Second, a conceive-execute gap: frontier LLMs can propose strong attack ideas or execute them, but not all at once. We mitigate this by decomposing attack construction into strategy generation, execution, and post-hoc trajectory refinement. Third, manual elicitation is costly to scale, which we address with our semi-automated red-teaming pipeline. Applied to BashArena, an AI control setting for tool-using coding agents, this pipeline produces MonitoringBench, a benchmark of 2,644 attack trajectories for evaluating monitor capabilities and failure modes. Our pipeline produces more diverse and stronger attacks: Opus-4.5 monitor's catch rate falls from 94.9\\% on elicited-only Opus attacks to 60.3\\% on our best refined attacks, with larger drops for several mid-tier monitors. Attacks optimized against three development monitors generalize to ten held-out monitors, with catch rates generally increasing with monitor capability. Using this benchmark, we provide a snapshot of the current monitor capabilities and find that frontier monitors often detect suspicious actions but fall for persuasion or fail to calibrate suspiciousness scores appropriately, suggesting tractable paths for improvement. MonitoringBench provides both a static benchmark for current tool-use monitors and a reusable methodology for refreshing these evaluations as agents and monitors improve.\nSubjects:\nCryptography and Security (cs.CR) ; Artificial Intelligence (cs.AI)\nCite as:\narXiv:2605.09684 [cs.CR]\n(or\narXiv:2605.09684v1 [cs.CR] for this version)\nhttps://doi.org/10.48550/arXiv.2605.09684\nFocus to learn more\narXiv-issued DOI via DataCite\nSubmission history\nFrom: Monika Jotautaitė [ view email ] [v1]\nSun, 10 May 2026 18:04:31 UTC (2,724 KB)\nFull-text links:\nAccess Paper:\nView a PDF of the paper titled MonitoringBench: Semi-Automated Red-Teaming for Agent Monitoring, by Monika Jotautait\\.e and 3 other authors\nView PDF\nHTML (experimental)\nTeX Source\nview license\nAdditional Features\nAudio Summary\nCurrent browse context:\ncs.CR\n< prev\n|\nnext >\nnew\n|\nrecent\n| 2026-05\nChange to browse by:\ncs\ncs.AI\nReferences & Citations\nSciX\nGoogle Scholar\nSemantic Scholar\nexport BibTeX citation\nLoading...\nBibTeX formatted citation\n×\nloading...\nData provided by:\nBookmark\nBibliographic Tools\nBibliographic and Citation Tools\nBibliographic Explorer Toggle\nBibliographic Explorer ( What is the Explorer? )\nConnected Papers Toggle\nConnected Papers ( What is Connected Papers? )\nLitmaps Toggle\nLitmaps ( What is Litmaps? )\nscite.ai Toggl","chars":3500,"total_chars":5202,"offset":0,"truncated":true,"next_offset":3500}