Prover-Estimator Debate: A New Scalable Oversight Protocol

Our new paper presents a protocol robust to this problem – but in order to prove that the protocol can in-principle answer any relevant question (‘completeness’), we need a stability assumption. The need for stability for completeness was discussed by Beth Barnes’ original post introducing obfuscated arguments ; our paper presents one route to stability, but does not resolve whether most questions have stable arguments in practice.

For anything the debaters know, there’s usually a sufficiently compact and stable honest argument such that the honest debater can win by giving this argument It seems pretty easy to give counterexamples here - e.g. Paul gives the example of unsupervised translation . But maybe these are relatively rare. It's plausible that the honest debaters in the human experiments I did where we ran into obfuscation / instability problems weren't using the right tactics (i.e. they could have constructed more stable arguments that used independent lines of evidence more)