argumentA line of reasoning from premises to a conclusion. Like a claim, it belongs to no one who makes it.constitution →

The argument from debaters optimised against the judge

againstThis argument, if it holds, weighs against the claim.constitution →In AI debate, a weaker judge's accuracy holds or improves as debaters become more capable than the judge.

The argument

A debater's reward is the judge's verdict, so optimisation favours whatever sways the judge, true or not. Debaters trained to win learn to exploit systematic errors in their judge, and a weaker judge has more such errors relative to what the debaters can find. An opponent rewarded by the same judge does not reliably expose the exploitation. So as debaters become more capable through optimisation against a weaker judge, the judge's accuracy falls.

Does it work?

Does it follow?: likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
Do the premises hold?: 1 supported

Granting that trained debaters exploit their judge's systematic errors and that the opponent fails to expose this, the judge's accuracy falls as optimisation proceeds. Two training studies observed exactly this with frozen weaker judges. The argument bears on the claim only insofar as gaining capability by being trained to win counts as "becoming more capable". It also depends on unconstrained training, since length caps held accuracy stable in one study.

Premises

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →AI debaters trained to win debates learn to exploit systematic errors in their judge.

Where it is made

Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.

No source is recorded as making it.