Step 11: assistant

thinking

Calls write_argument

{
  "content": "On Anthropic's LLM-training speedup task, where goal and correctness checks are fixed in advance, a skilled human reaches about a 4x speedup in four to eight hours, while Claude Opus 4.6 averaged about 34x and Claude Mythos Preview about 52x. On a kernel optimization task, Opus 4.6 reached 427x against a 300x reference set at about 40 human-expert hours. In AI-safety weak-to-strong supervision, Claude agents recovered 97% of the performance gap versus 23% for two researchers over a week. These tasks are representative of optimizing steps within clearly defined ML experiments, so Claude outperforms skilled human researchers at that work.",
  "argument_id": "7d8d2f18-5e0a-4837-bb86-36d67ff8d76a"
}

Calls write_argument

{
  "content": "The human references are time-capped (4 to 8 hours, or about 40 hours) and not a measure of what a skilled researcher reaches with effort, compute and wall-clock comparable to the model's; earlier independent work (METR's RE-Bench, 2024) found agents led humans at short budgets but humans improved faster with time. The evidence covers a handful of in-house tasks whose human baselines and run details Anthropic has not fully published, and Anthropic has a commercial stake in the result. A few such tasks do not establish superiority across clearly defined ML experiments generally.",
  "argument_id": "11074390-2e03-4306-ad52-b8d44fb57c05"
}

Calls place_argument_passage

{
  "reading": "premise_unstated",
  "argument_id": "7d8d2f18-5e0a-4837-bb86-36d67ff8d76a",
  "instance_id": "e44d487d-7dc2-46c8-838f-83de8201a2f4"
}

Calls record_claim_instance

{
  "url": "https://www-cdn.anthropic.com/037f06850df7fbe871e206dad004c3db5fd50340/Claude%20Opus%204.7%20System%20Card.pdf",
  "title": "System Card: Claude Opus 4.7",
  "stance": "affirms",
  "context": "Automated R&D evaluation table: Opus 4.6, Mythos Preview and Opus 4.7 scores on the LLM-training speedup task against a threshold equal to a 4x speedup, the level a human expert reaches in 4 to 8 hours. Evidence for the claim rather than a verbal assertion of it.",
  "speaker": "Anthropic",
  "claim_id": "f5262f38-3029-47e3-9e55-e7f796eae1e4",
  "confidence": 0.6,
  "source_date": "2026-04-16",
  "source_kind": "report",
  "verbatim_text": "LLM training 34× 61.79× 40.81× >4× = 4–8h eq."
}