No proven general method exists for training AI on tasks too complex for humans to demonstrate or evaluate.
Assessed
Kept the existing two-argument structure, which reflects the question well. Assessed the claim as supported (credence 0.85) on the record of scalable-oversight research. Rated the argument for as likely and the argument against as slight. Set importance to 0.45 and contestation to 0.3. Did not notify anyone, because the claim has no dependents.
Assessed Supported
verdict confidence 0.75 · credence 0.85
The claim is that no training method has been shown to work in general for tasks that humans can neither perform nor reliably judge. As of late 2026 the evidence favors it. Standard methods such as imitation learning and reinforcement learning from human feedback rely on human demonstrations or human evaluations, and so are bounded by them. The proposals meant to get past that limit include iterated amplification, AI safety via debate, recursive reward modeling and weak-to-strong generalization. All of them have been tested only in proxy settings: tasks where experimenters know the ground truth, and where the gap between supervisor and model is artificial or small. In those settings the results show partial and uneven gains over unaided weak supervision. Weak-to-strong experiments recover only part of the strong model's capability, and debate helps weak judges on some tasks but not on others. No method has been shown to produce a reliable training signal on a task that is truly beyond human evaluation. The major labs and the research agendas behind these methods themselves describe the problem as unsolved. The main room for disagreement is over the word "proven." On a loose reading, any of these techniques that does better than plain human supervision in a sandwiching or weak-supervisor experiment could count as a working general method, and labs already use AI-assisted feedback in training. On the ordinary reading, though, partial gains in proxy setups do not show that a method works generally when humans cannot check the result. A demonstration of reliable oversight on a real superhuman task, or a general guarantee for one of these protocols, would change the verdict.
Claim entered the graph