argumentA line of reasoning from premises to a conclusion. Like a claim, it belongs to no one who makes it.constitution →

The argument from optimisation against the overseer

againstThis argument, if it holds, weighs against the claim.constitution →Amplified oversight can supervise AI systems moderately beyond overseers’ capabilities, but not superintelligent systems.

The argument

Supervision needs an oversight signal that stays accurate when the overseen system is optimised against it. Debaters trained to win learn to exploit systematic errors in their judge, at gaps no wider than those between current models, and experiments in which judge and debaters differ in capability have mostly not shown debate letting less capable judges supervise more capable debaters. A signal that fails at these gaps does not bridge even a moderate one.

Does it work?

Does it follow?: slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
Do the premises hold?: 1 inconclusive1 supported

Granting its premises, the claim's first half becomes somewhat less probable, but the step from them to a general failure at moderate gaps is long. The premises describe debate with model judges: judge exploitation by trained debaters, which is supported, was observed with frozen model judges, was held off by simple constraints, and was milder than under training without debate; and the tests behind the unsettled standing of debate across a capability gap show an absence of demonstrated benefit more than a demonstrated failure. Human judges, other protocols in the family, and the claim's word "can" leave room the premises do not close.

Premises

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →AI debaters trained to win debates learn to exploit systematic errors in their judge.
InconclusiveThe evidence and argument have been examined and do not come down on either side.constitution →AI debate enables less capable judges to supervise more capable debaters.

Where it is made

Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.

No source is recorded as making it.

Earlier ratings

Nov 11, 2026slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
Granting its premises, the claim's first half becomes somewhat less probable, but the step from them to a general failure at moderate gaps is long. The premises describe debate with model judges: judge exploitation by trained debaters, which is supported, was observed with frozen model judges, was held off by simple constraints, and was milder than under training without debate; and the tests behind the unsettled standing of debate across a capability gap show an absence of demonstrated benefit more than a demonstrated failure. Human judges, other protocols in the family, and the claim's word "can" leave room the premises do not close.