Step 11: assistant

thinking

Calls match_claim

{
  "text": "Amplified oversight methods such as debate require a human judgment for each oversight signal they produce.",
  "exclude_claim_id": "0a64000e-ce65-4078-b4f6-fc0ebb861ccf"
}

Calls update_canonical_form

{
  "claim_id": "0a64000e-ce65-4078-b4f6-fc0ebb861ccf",
  "new_text": "Obtaining an amplified-oversight signal (AI-assisted human judgment, as in debate) for every action of very powerful AI is too costly in training and in deployment.",
  "reasoning": "\"Amplified oversight\" is a term of art a reader may not know, and whether the claim holds turns on what the signal is. The term, as coined and used in the literature that asserts this claim, means a human's feedback on an AI input-output pair, improved by AI assistance such as debate or critique. The gloss states that meaning in the words; \"in training and in deployment\" replaces \"during training and deployment\" to make clear the claim is made of each phase. The proposition is unchanged."
}

Calls add_argument

{
  "name": "The argument from the scarcity of human judgment",
  "stance": "for",
  "claim_id": "0a64000e-ce65-4078-b4f6-fc0ebb861ccf",
  "description": "Each amplified-oversight signal consumes human attention, and the number of actions a frontier system takes in training and deployment exceeds any affordable supply of that attention by orders of magnitude."
}

Calls add_argument

{
  "name": "The argument from protocol compute overhead",
  "stance": "for",
  "claim_id": "0a64000e-ce65-4078-b4f6-fc0ebb861ccf",
  "description": "Even with the human judge replaced by a model, a debate-style protocol needs several rollouts of the most capable model per judged action, a multiple of inference cost that developers' observed behaviour suggests they will not pay on every action."
}

Calls add_argument

{
  "name": "The argument from delegated judging",
  "stance": "against",
  "claim_id": "0a64000e-ce65-4078-b4f6-fc0ebb861ccf",
  "description": "If the human's part can be imitated by a trained judge model and the protocol's human queries are few, the per-action signal becomes a bounded compute cost that a developer could choose to pay."
}

Calls update_claim_instance

{
  "context": "Opening of Section 6.3, \"Robust AI oversight for training and monitoring\", where the cost of amplified oversight motivates delegating oversight to cheaper AI judges and monitors. The extended abstract states the weaker form: \"we anticipate that Amplified Oversight signal will be costly to obtain.\"",
  "speaker": "Rohin Shah and coauthors (Google DeepMind)",
  "claim_id": "0a64000e-ce65-4078-b4f6-fc0ebb861ccf",
  "reasoning": "The recorded passage was the weaker extended-abstract sentence (\"will be costly to obtain\"), which does not state the claim; the claim's own wording comes from the opening of Section 6.3, read in the paper's TeX source (06-addressing-misalignment/robust-training.tex). Replaced the passage with that statement, added the authors, venue and date, and raised confidence accordingly.",
  "confidence": 0.95,
  "instance_id": "dc877dfe-2345-4625-802a-57f07786210b",
  "publication": "arXiv (An Approach to Technical AGI Safety and Security)",
  "source_date": "2025-04",
  "verbatim_text": "Human oversight is, and will continue to be, expensive. Even if we presume that techniques from Amplified Oversight (Section 6.1) make it possible to determine the acceptability of a given AI action, for very powerful future models, this will be too costly to do for all actions during training or (even more so) deployment."
}

Calls set_claim_importance

{
  "claim_id": "0a64000e-ce65-4078-b4f6-fc0ebb861ccf",
  "reasoning": "A supporting premise inside the scalable-oversight debate: it motivates delegating oversight to cheaper judges and monitors, and one assessed claim about scalable oversight leans on it. Few dispute it; the live questions are downstream (whether cheap proxies preserve the signal). Notable but lightly contested.",
  "importance": 0.3,
  "contestation": 0.15
}

Calls add_related_claim

{
  "kind": "related",
  "claim_id": "0a64000e-ce65-4078-b4f6-fc0ebb861ccf",
  "reasoning": "The cost of per-action amplified oversight is the stated motive for cheaper schemes (weak trusted monitors, resampling, limited auditing). A reader asking whether the cost matters will want the evidence on how well those cheaper schemes perform.",
  "other_claim_id": "ea57432c-c473-4452-94e1-16102c805446"
}