Step 7: assistant

thinking

Calls update_claim_assessment

{
  "status": "supported",
  "claim_id": "5569bcb9-dfbe-4b80-ab83-df19547fd1c4",
  "assessment": "The claim is that no training method has been shown to work in general for tasks that humans can neither perform nor reliably judge. As of late 2026 the evidence favors it. Standard methods such as imitation learning and reinforcement learning from human feedback rely on human demonstrations or human evaluations, and so are bounded by them. The proposals meant to get past that limit include iterated amplification, AI safety via debate, recursive reward modeling and weak-to-strong generalization. All of them have been tested only in proxy settings: tasks where experimenters know the ground truth, and where the gap between supervisor and model is artificial or small. In those settings the results show partial and uneven gains over unaided weak supervision. Weak-to-strong experiments recover only part of the strong model's capability, and debate helps weak judges on some tasks but not on others. No method has been shown to produce a reliable training signal on a task that is truly beyond human evaluation. The major labs and the research agendas behind these methods themselves describe the problem as unsolved.\n\nThe main room for disagreement is over the word \"proven.\" On a loose reading, any of these techniques that does better than plain human supervision in a sandwiching or weak-supervisor experiment could count as a working general method, and labs already use AI-assisted feedback in training. On the ordinary reading, though, partial gains in proxy setups do not show that a method works generally when humans cannot check the result. A demonstration of reliable oversight on a real superhuman task, or a general guarantee for one of these protocols, would change the verdict.",
  "confidence": 0.75,
  "claim_credence": 0.85,
  "marginal_yield": 0.25,
  "reasoning_trace": "My starting point was what the claim would need. For \"no proven general method\" to be false, some technique would have to be shown to yield a reliable training signal on tasks humans cannot demonstrate or evaluate, across task types.\n\nEvidence considered, from background knowledge of the literature, since no new sources were opened in this pass:\n- Christiano, Shlegeris & Amodei (2018), \"Supervising strong learners by amplifying weak experts\" (arXiv 1810.08575). This is the one recorded source. It motivates amplification on the grounds that human training signals fail for tasks beyond human scale, and it tests amplification only on algorithmic toy tasks where ground truth is known.\n- Irving, Christiano & Amodei (2018), on debate. Later debate experiments (Michael et al. 2023; Khan et al. 2024; Kenton et al. 2024) found gains for weak judges on reading-comprehension tasks with information asymmetry, mixed results elsewhere, and all of them used tasks with known answers.\n- Bowman et al. (2022), on sandwiching. Here AI assistance improved non-expert performance on tasks where expert labels were available.\n- Burns et al. (2023), on weak-to-strong generalization. Performance recovered only partially, varied across tasks, and was much weaker for reward modeling.\n- Lab statements, including OpenAI's 2023 Superalignment announcement and DeepMind's 2025 AGI-safety approach, which treat amplified oversight as an open research problem.\n\nWeighing: the argument from unvalidated proposals goes through on the ordinary reading of \"proven.\" The argument from existing candidate methods shows that candidate methods exist and give some gains, but not that any has been proven general. The residual doubt comes from how loosely \"proven\" can be read, and from the chance that 2025–2026 work I have not examined shows something stronger. That is why the status is supported rather than verified, with credence 0.85."
}