Step 10: tool results

{
  "name": "The argument from checkable outputs and everyday review",
  "stance": "for",
  "claim_id": "353f02bc-ebdf-443f-a09e-38209d365bd4",
  "description": "Where a solution can be tested against a specification or certificate, and in ordinary delegated work, checking takes far less than producing; AI outputs are taken to be of this kind."
}

{
  "name": "The argument from protocols that shift the work onto the producer",
  "stance": "for",
  "claim_id": "353f02bc-ebdf-443f-a09e-38209d365bd4",
  "description": "Debate, monitoring and related protocols make strong systems expose or contest each other's flaws, leaving a weaker checker a task within its reach."
}

{
  "name": "The argument from the universal quantifier under adversarial pressure",
  "stance": "against",
  "claim_id": "353f02bc-ebdf-443f-a09e-38209d365bd4",
  "description": "Certifying safety means ruling out every flaw, while a producer optimizing against the check needs only one flaw the checker cannot see; so safety checking is not the cheap side of the asymmetry."
}

{
  "name": "The argument from measured dependence of detection on checker capability",
  "stance": "against",
  "claim_id": "353f02bc-ebdf-443f-a09e-38209d365bd4",
  "description": "In sabotage and backdoor experiments, how much concealed misbehavior is caught tracks the checker's capability relative to the producer's, and weak checkers and unaided humans miss most well-hidden flaws."
}

{
  "claim_id": "353f02bc-ebdf-443f-a09e-38209d365bd4",
  "reasoning": "This is the premise behind scalable oversight, AI control with trusted weaker monitors, and plans to delegate alignment research to AI, and it is a named point of disagreement between Christiano on one side and Yudkowsky and Wentworth on the other, with active empirical work in 2025 and 2026. Consequential within AI safety and actively argued, but narrower in reach than the central claims it feeds (about on a level with 'AI debate enables less capable judges to supervise more capable debaters', below 'Scalable oversight can reliably control superintelligent AI systems').",
  "importance": 0.6,
  "contestation": 0.7
}

{
  "url": "https://www.lesswrong.com/posts/CoZhXrhpQxpy9xw9y/where-i-agree-and-disagree-with-eliezer",
  "title": "Where I agree and disagree with Eliezer",
  "stance": "affirms",
  "context": "One of a numbered list of disagreements with Yudkowsky's 'AGI Ruin: A List of Lethalities'; here replying to the claim that humans could not verify plans or research proposed by AI systems.",
  "speaker": "Paul Christiano",
  "claim_id": "353f02bc-ebdf-443f-a09e-38209d365bd4",
  "confidence": 0.85,
  "publication": "LessWrong / AI Alignment Forum",
  "source_date": "2022-06",
  "verbatim_text": "Eliezer seems to argue that humans couldn’t verify pivotal acts proposed by AI systems (e.g. contributions to alignment research), and that this further makes it difficult to safely perform pivotal acts. In addition to disliking his concept of pivotal acts, I think that this claim is probably wrong and clearly overconfident. I think it doesn’t match well with pragmatic experience in R&D in almost any domain, where verification is much, much easier than generation in virtually every domain."
}

{
  "url": "https://www.lesswrong.com/posts/2PDC69DDJuAx6GANa/verification-is-not-easier-than-generation-in-general",
  "title": "Verification Is Not Easier Than Generation In General",
  "stance": "denies",
  "context": "A short post arguing that the intuition that checking is easier than producing is drawn from a class of problems selected for easy checking; in the comments the author applies it to adversarial settings and to verifying that software is secure.",
  "speaker": "John Wentworth",
  "claim_id": "353f02bc-ebdf-443f-a09e-38209d365bd4",
  "confidence": 0.85,
  "publication": "LessWrong / AI Alignment Forum",
  "source_date": "2022-12-06",
  "verbatim_text": "I think a more accurate takeaway would be that among problems in NP, verification is easier than generation. In other words, among problems for which verification is easy, verification is easier than generation. Rather a less impressive claim, when you put it like that. With that in mind, here is an algorithmic problem for which generation is easier than verification. Predicate: given a program, does it halt? Generation problem: generate a program which halts. Verification problem: given a program, verify that it halts. The generation problem is trivial. The verification problem is uncomputable."
}

{
  "url": "https://www.lesswrong.com/posts/2PDC69DDJuAx6GANa/verification-is-not-easier-than-generation-in-general",
  "title": "Verification Is Not Easier Than Generation In General (comment thread)",
  "stance": "affirms",
  "context": "Reply to Wentworth's post, defending the everyday basis of the asymmetry for delegation to AI while conceding it has exceptions. A qualified affirmation.",
  "speaker": "Paul Christiano",
  "claim_id": "353f02bc-ebdf-443f-a09e-38209d365bd4",
  "confidence": 0.7,
  "publication": "LessWrong / AI Alignment Forum",
  "source_date": "2022-12-06",
  "verbatim_text": "I think most people’s intuitions come from more everyday experiences like: It’s easier to review papers than to write them. Fraud is often caught using a tiny fraction of the effort required to perpetrate it. I can tell that a piece of software is useful for me more easily than I can write it. These observations seem relevant to questions like “can we delegate work to AI” because they are ubiquitous in everyday situations where we want to delegate work. [...] I do think there are plenty of examples where verification is not easier than generation (and certainly where verification is non-trivial). It’s less clear what the relevance of that is."
}