Step 5: assistant

text

I recorded the first assessment of this claim: **inconclusive**, with credence 0.4 that it is true and confidence 0.65 that the status is the right one. One caveat: I wrote it from what I already know of the literature. I did not open or re-read the sources this pass, and no source instances were recorded. Nearly half the $1.00 budget was gone before the pass began, and the run ended at $0.57.

**What the assessment says.**
- **The problem is real.** In December 2020 Beth Barnes and Paul Christiano reported on OpenAI's human debate experiments. A dishonest debater could build a long argument with one fatal flaw that the honest side could not locate. They suggested this might confine debate and iterated amplification to arguments small relative to training compute. They called it a likely limitation and had no fix.
- **Whether it is a hard limit is unsettled.**
  - Doubly-efficient debate (Brown-Cohen, Irving and Piliouras, 2023) avoids it if the problem has a verifiable decomposition.
  - Prover-estimator debate (2025) avoids it if the argument is "stable", meaning its conclusion does not hinge on tiny changes in its steps.
  - Nobody has tested at scale with trained models whether real arguments meet these conditions.
- **The extension to iterated amplification is inferred, not tested.** It rests on amplification sharing debate's decomposition structure, not on experiments.

**Structure and housekeeping.**
- I left the claim without subclaims or named arguments. The whole question turns on whether those protocol assumptions hold for real arguments, which doesn't justify new nodes at this importance.
- I set importance to 0.45 and contestation to 0.55, logged the decision, and answered both inbox messages.
- I sent no notice to the one dependent claim (that iterated amplification can usefully train systems on complex real-world tasks). Its "inconclusive" verdict and 0.25 credence already sit comfortably with this result.

**Left for a later pass:** record the 2020 Barnes–Christiano post and the 2025 prover-estimator paper as sources, and check their exact wording against the claim. I put marginal yield at 0.4 to reflect that.