Step 16: tool results

read_page

{
  "url": "https://blog.redwoodresearch.org/p/how-might-we-safely-pass-the-buck",
  "offset": 14000,
  "max_chars": 5500
}

{"ok":true,"url":"https://blog.redwoodresearch.org/p/how-might-we-safely-pass-the-buck","text":"le can’t delegate tasks to collaborators who share their skills and context.\nAI systems will have strictly more skills and context than humans have. Imagine you can delegate research tasks to the best human researchers you know . You might trust these researchers even more than you trust yourself.\nDevelopers will be in a similar position with AI. Researchers will be proven wrong over and over again by their AI agents – to the point where it's clear the AI agents are more competent than them. See section 6 for other evidence that could justify the capability condition.\nAI agents might have very little time to scale safety to superhuman capabilities. So humans should spend most of their effort on scalable safety, rather than ‘passing the buck.’\nI address this objection in section 3.\nWe have much more data on humans than on AI, so our standards of evidence for AI safety should be higher than they are for human safety.\nI agree. It’s hard to know if AI agents have some flaw that’s hard to notice (e.g. AI agents don’t care about how their actions affect the broader world), that might be catastrophic.\nHowever, I’m not arguing that developers should apply the same safety checks to AI systems as the ones that are applied to humans. I’m arguing that AI agent behavior needs to be as safe as human behavior during a specific deferred task.\nThe checks we perform on AI should probably be much more strenuous than those we perform on humans.\nThis perspective does not account for the benefits of buck passing for speeding up (responsible) AI development.\nThat’s correct. I only consider whether passing the buck minimizes risks from a developer's own AI systems . I don’t perform a complete cost-benefit analysis .\nAutonomous control seems like a stack of cards. I agree. I’d much rather justify buck-passing with alignment; however, autonomous control might be helpful as an extra layer of defense, and is worth investigating further.\nIf you still have objections or confusions, they might be answered in the remainder of the post.\n2. What I mean by “passing the buck” to AI\nI don’t believe there is a discrete moment when “humans are mostly in the loop” and then “humans are mostly out of the loop.”\nInstead, the involvement of human researchers will decrease continuously (figure 12).\nThere are multiple places along this continuous decline of human technical influence where one could say an AI company has “passed the buck” to AI.\nFigure 12. The importance of human researchers for safety declines continuously until full obsolescence.\nImagine that you are an AI researcher, and AI agents perform 10,000 4 hour research tasks every 24 hours. You might still provide oversight in the sense that you have to hit the ‘approve’ button on deployments, but eventually, it’s clear you are a rubber stamp. You cannot keep up. If the agents were trying to compromise your research cluster or manufacture fake safety results, you would not know.\nThus, even if human researchers still have influence on paper , developers may have de facto passed the buck before then.\nI’ll say a developer ‘passed the buck to AI’ if (1) AI agents perform most internal AI software R&D and (2) human oversight over AI agents has little bearing on the safety of the situation.\nAfter this threshold, the primary approach for ensuring safety changes. It changes from “humans will notice if something fishy is going on and correct it” to some other approach that allows for weak human oversight.\nThis post explores what this alternative approach to ensuring safety might be.\n3. Why focus on “passing the buck” rather than “aligning superintelligence.”\nI imaging many readers might object to the “pass the buck” framing. Isn’t the ultimate goal of technical safety to scale alignment to superintelligence? Yes, developing safe superintelligence is my ultimate goal; however, my primary instrumental focus is to pass the buck to AI. Before explaining why passing the buck is a good intermediate to focus on, I should first justify why ‘safe superintelligence’ is the ultimate target I’m aiming for.\nI’m concerned about superintelligence because I think the most extreme risks will come from the most capable systems, and I believe superintelligent systems could plausibly emerge soon. For example, superintelligence might end the lives of billions of people with a novel bioweapon , or permanently concentrate geopolitical power into the hands of a handful of tech or political bosses. I believe these risks are plausible and proximal such that they dominate the risks from weaker AI systems.\nTherefore, insofar as the task of AI safety researchers is to “make AI safe,” I believe the top priority should be to make superintelligence safe.\nHowever, the best path to achieving this goal might not be the most direct.\nThere’s a direct path where humans do all the work to create safe superintelligence.\nThere’s also an indirect path where humans pass the buck to AI systems that finish the job.\nFigure 13. Two paths to superintelligence.\nThe indirect path is probably easier than the direct one. Weaker AI systems are easier to evaluate than superintelligent ones , so it's easier to train them to behave as intended than it is to train superintelligent agents to do what we want. This argument suggests humans should pass the buck to AI agents eventually ; however, it does not imply that passing the buck should be the primary target of preparation . Alternatively, human researchers might try to give their","chars":5500,"total_chars":57495,"offset":14000,"truncated":true,"next_offset":19500}