Run step
Step 9: assistant
thinking
Calls code_execution
{
"code": "\nshow(2,[0,2],1300); show(5,[0],1200)\n"
}code_execution_tool_result
{
"type": "code_execution_tool_result",
"content": {
"type": "code_execution_result",
"stderr": "",
"stdout": "## https://arxiv.org/pdf/2312.06942\nis standing in for the developers of an AI system) proposes a strategy we call *trusted monitoring*,\n\nwhere we monitor GPT-4’s outputs with a trusted but less capable model, GPT-3.5.\n\nIn response to trusted monitoring, the red-team instructs GPT-4 to look for good opportunities to\n\nmuch better than trusted monitoring.\n\nsafety despite intentional subversion.\n\nTrusted monitoring: Produce code using U, and · 97% · Write code with a subtle back- · 62%\n\n| Successful backdoor attempt | 6 | 13 |\n| Failed backdoor attempt | 18 | 34 |\n\n*Safety*: The blue team win probability is 62% (up from 15% when using upfront auditing)\n\nmonitoring.\n\nTrusted monitoring performs better than upfront auditing for two reasons:\n\n• Firstly, trusted monitoring increases the probability with which an attempted backdoor will\n\n• Secondly, trusted monitoring forces the attacker to use much sneakier backdoors–it can\n\nlow success rate means that the trusted monitoring has (on average) many opportunities\n\nThis means that trusted monitoring\n\nFigure 5: Defer to trusted results for untrusted monitoring.\n\n• *Trusted model adversarial evaluation of the policy:* Using GPT-3.5 to generate large numbers of APPS problems, running the policy on them, then using trusted monitoring to look\n\nGPT-4 to try to jailbreak GPT-3.5.\n\n•\n\n## https://arxiv.org/html/2312.06942v5\n… Safety: **The blue team win probability is 62%** (up from 15% when using upfront auditing)\n\n# AI Control: Improving Safety Despite Intentional Subversion\n\nUse trusted monitoring to determine suspicion.\n\n### 3.2 Trusted Monitoring\n\nSafety: The blue team win probability is 62% (up from 15% when using upfront auditing)\n\n- • Firstly, trusted monitoring increases the probability with which an attempted backdoor will be audited.\n\nThis low success rate means that the trusted monitoring has (on average) many opportunities to catch an attempted backdoor before one succeeds.\n\n| Trusted monitoring; defer to trusted 0% | 62% | 97% |\n| Trusted monitoring; defer to trusted 5% | 74% | 95% |\n| Trusted monitoring; defer to trusted 10% | 83% | 93% |\n| Trusted monitoring; defer to trusted 20% | 92% | 83% |\n| Trusted monitoring; defer to trusted 50% | 100% | 51% |\n\n| Trusted monitoring; defer to trusted 0% | 62% | 97% |\n| Trusted monitoring; defer to trusted 5% | 74% | 95% |\n| Trusted monitoring; defer to trusted 10% | 83% | 93% |\n| Trusted monitoring; defer to trusted 20% | 92% | 83% |\n| Trusted monitoring; defer to trusted 50% | 100% | 51% |\n| Untrusted monitoring; defer to trusted 0% | 79% | 96% |\n| Untrusted monitoring; defer to trusted 5% | 82% | 95% |\n| Untrusted monitoring; defer to trusted\n\n## https://www.lesswrong.com/posts/7fJRPB6CF6uPKMLWi/my-ai-model-delta-compared-to-christiano\nI put approximately-zero probability on the possibility that Paul is basically right on this delta; I think he’s completely out to lunch.\n\n# 196\n## My AI Model Delta Compared To Christiano\nOn my models, HCH adds *recursion* to the universal pernicious difficulties of delegation, and my main response is to run away screaming. But on Paul’s models, delegation is fundamentally viable, so why *not* delegate recursively?\n\nThat accounts for most of the known-to-me places where my models differ from Paul’s.\n\n# 196\nI think both that:\n\n- this is not a good characterization of Paul's views\n- verification is typically easier than generalization and this fact is important for the overall picture for AI risk\n\n- the motte: there exist hard to verify properties\n\nI don’t think Paul thinks verification is generally easy or that delegation is fundamentally viable.\n\n- Verification vs generation has an extremely wide space of possible interpretations, and as stated here the claim is incredibly vague. The argument for why difficulty of verification implies difficulty of delegation is not laid out, and the examples do not go in much depth.\n\nBut that doesn't mean generation is as hard as verification in \n\n",
"content": [],
"return_code": 0,
"abort_reason": null
},
"tool_use_id": "srvtoolu_0141B3fTbNvuUonHKQJUtY2M"
}thinking
Calls add_argument
{
"name": "The argument from checkable outputs and everyday review",
"stance": "for",
"claim_id": "353f02bc-ebdf-443f-a09e-38209d365bd4",
"description": "Where a solution can be tested against a specification or certificate, and in ordinary delegated work, checking takes far less than producing; AI outputs are taken to be of this kind."
}Calls add_argument
{
"name": "The argument from protocols that shift the work onto the producer",
"stance": "for",
"claim_id": "353f02bc-ebdf-443f-a09e-38209d365bd4",
"description": "Debate, monitoring and related protocols make strong systems expose or contest each other's flaws, leaving a weaker checker a task within its reach."
}Calls add_argument
{
"name": "The argument from the universal quantifier under adversarial pressure",
"stance": "against",
"claim_id": "353f02bc-ebdf-443f-a09e-38209d365bd4",
"description": "Certifying safety means ruling out every flaw, while a producer optimizing against the check needs only one flaw the checker cannot see; so safety checking is not the cheap side of the asymmetry."
}Calls add_argument
{
"name": "The argument from measured dependence of detection on checker capability",
"stance": "against",
"claim_id": "353f02bc-ebdf-443f-a09e-38209d365bd4",
"description": "In sabotage and backdoor experiments, how much concealed misbehavior is caught tracks the checker's capability relative to the producer's, and weak checkers and unaided humans miss most well-hidden flaws."
}Calls set_claim_importance
{
"claim_id": "353f02bc-ebdf-443f-a09e-38209d365bd4",
"reasoning": "This is the premise behind scalable oversight, AI control with trusted weaker monitors, and plans to delegate alignment research to AI, and it is a named point of disagreement between Christiano on one side and Yudkowsky and Wentworth on the other, with active empirical work in 2025 and 2026. Consequential within AI safety and actively argued, but narrower in reach than the central claims it feeds (about on a level with 'AI debate enables less capable judges to supervise more capable debaters', below 'Scalable oversight can reliably control superintelligent AI systems').",
"importance": 0.6,
"contestation": 0.7
}Calls record_claim_instance
{
"url": "https://www.lesswrong.com/posts/CoZhXrhpQxpy9xw9y/where-i-agree-and-disagree-with-eliezer",
"title": "Where I agree and disagree with Eliezer",
"stance": "affirms",
"context": "One of a numbered list of disagreements with Yudkowsky's 'AGI Ruin: A List of Lethalities'; here replying to the claim that humans could not verify plans or research proposed by AI systems.",
"speaker": "Paul Christiano",
"claim_id": "353f02bc-ebdf-443f-a09e-38209d365bd4",
"confidence": 0.85,
"publication": "LessWrong / AI Alignment Forum",
"source_date": "2022-06",
"verbatim_text": "Eliezer seems to argue that humans couldn’t verify pivotal acts proposed by AI systems (e.g. contributions to alignment research), and that this further makes it difficult to safely perform pivotal acts. In addition to disliking his concept of pivotal acts, I think that this claim is probably wrong and clearly overconfident. I think it doesn’t match well with pragmatic experience in R&D in almost any domain, where verification is much, much easier than generation in virtually every domain."
}Calls record_claim_instance
{
"url": "https://www.lesswrong.com/posts/2PDC69DDJuAx6GANa/verification-is-not-easier-than-generation-in-general",
"title": "Verification Is Not Easier Than Generation In General",
"stance": "denies",
"context": "A short post arguing that the intuition that checking is easier than producing is drawn from a class of problems selected for easy checking; in the comments the author applies it to adversarial settings and to verifying that software is secure.",
"speaker": "John Wentworth",
"claim_id": "353f02bc-ebdf-443f-a09e-38209d365bd4",
"confidence": 0.85,
"publication": "LessWrong / AI Alignment Forum",
"source_date": "2022-12-06",
"verbatim_text": "I think a more accurate takeaway would be that among problems in NP, verification is easier than generation. In other words, among problems for which verification is easy, verification is easier than generation. Rather a less impressive claim, when you put it like that. With that in mind, here is an algorithmic problem for which generation is easier than verification. Predicate: given a program, does it halt? Generation problem: generate a program which halts. Verification problem: given a program, verify that it halts. The generation problem is trivial. The verification problem is uncomputable."
}Calls record_claim_instance
{
"url": "https://www.lesswrong.com/posts/2PDC69DDJuAx6GANa/verification-is-not-easier-than-generation-in-general",
"title": "Verification Is Not Easier Than Generation In General (comment thread)",
"stance": "affirms",
"context": "Reply to Wentworth's post, defending the everyday basis of the asymmetry for delegation to AI while conceding it has exceptions. A qualified affirmation.",
"speaker": "Paul Christiano",
"claim_id": "353f02bc-ebdf-443f-a09e-38209d365bd4",
"confidence": 0.7,
"publication": "LessWrong / AI Alignment Forum",
"source_date": "2022-12-06",
"verbatim_text": "I think most people’s intuitions come from more everyday experiences like: It’s easier to review papers than to write them. Fraud is often caught using a tiny fraction of the effort required to perpetrate it. I can tell that a piece of software is useful for me more easily than I can write it. These observations seem relevant to questions like “can we delegate work to AI” because they are ubiquitous in everyday situations where we want to delegate work. [...] I do think there are plenty of examples where verification is not easier than generation (and certainly where verification is non-trivial). It’s less clear what the relevance of that is."
}