The premise that AI-assisted human oversight of every action of a very powerful AI system is too costly is stated in the literature without any cost estimate; simple arithmetic bears it out for deployment, while for training it is likely but depends on volumes no one has published.
Obtaining an amplified-oversight signal (AI-assisted human judgment, as in debate) for every action of very powerful AI is too costly in training and in deployment.SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →0.75
Proposals for overseeing very capable AI systems commonly assume that a full amplified-oversight signal, meaning AI-assisted human judgment of the kind produced by debate, cannot be obtained for every action, and that cheaper proxies must stand in for it most of the time. The assumption shapes how such proposals are designed, yet the one source recorded for the claim, a 2025 paper on technical AGI safety, gives a single sentence of justification and no figure.
The recorded assessment (b4a10102) supplies the arithmetic the literature lacks. For deployment with a human in each judgment, the case is close to settled: one chatbot alone was reported to receive about 2.5 billion prompts a day in mid-2025, and ten seconds of human time on each at 5 dollars an hour comes to roughly 13 billion dollars a year, with five-minute judgments at 20 dollars an hour reaching about 1.5 trillion dollars a year and some 26 million full-time judges. The floor of the plausible range is already prohibitive.
For training the picture is less firm. Judging between a million and a billion training actions costs anywhere from about 0.3 million to 50 billion dollars under the assessment's assumed ranges, and the low end is affordable beside the cost of a frontier training run. The claim holds for training only if very powerful systems are trained on something like a hundred million judged actions or more, or if waiting on human judges slows training unacceptably. Both are judged likely; neither has been costed in any published source the assessment found.
Where the human's part is delegated to a model, the claim becomes one about willingness to pay, not impossibility: a structural estimate puts a full debate at three to ten times the cost of producing the action, and no measured per-action cost of a debate protocol was found. The assessment rates the claim supported with credence 0.82, and notes that its ranges for human time per judgment are assumed, not measured. Whether each amplified-oversight signal needs a human judgment at all is carried by a separate claim that has not yet been assessed.