Step 4: tool results

get_claim

{
  "claim_id": "ec9c31d9-8ba5-4107-86f2-25ee089e89a1"
}

{"claim":{"id":"ec9c31d9-8ba5-4107-86f2-25ee089e89a1","kind":"claim","canonical_form":"Anthropic's models have improved on capability benchmarks about twice as fast since October 2025 as from March 2024 to October 2025.","claim_type":"empirical_derived","state":"active","decomposition_status":"complete","importance":0.45,"contestation":0.4,"domains":[],"tags":[{"slug":"anthropic","name":"Anthropic"},{"slug":"llm-capability-trends","name":"LLM capability trends"},{"slug":"ai-capability-evaluation","name":"AI Capability Evaluation"}],"steward_state":"done","stewarded_at":"2026-11-12T00:19:31.808Z","canonical_direction_note":null,"created_at":"2026-11-11T23:24:16.443Z","updated_at":"2026-11-12T00:19:52.042Z","page_url":"https://minerval.ai/claims/ec9c31d9-8ba5-4107-86f2-25ee089e89a1"},"assessment":{"id":"963d7dd4-90d4-464e-9f38-9e23871fc528","status":"supported","confidence":0.55,"claim_credence":0.65,"summary":"The claim rests mainly on METR's assessment of 5 November 2026, which had access to Anthropic's internal benchmark scores, including unreleased models. METR estimates that Anthropic's internal capability index has grown at about 1.9 times its earlier rate since October 2025, with a 95% interval of 1.5 to 2.8. Its robustness checks span 1.1 to 2.9, and the low end comes from changing the link function. Anthropic's own Haiku 5.5 system card reports a factor of 1.7. An outside analysis reportedly gets about 1.6 using only public data on released Claude models.\n\nThe evidence shows a speed-up of some size fairly well. The specific figure of \"about twice\" is less firm. The estimates run from roughly 1.6 to 1.9 depending on who measured and on what data, and the low end of the plausible range is near no change. Several things call for caution. The data points are sparse. The breakpoint was fitted, so some regression to the mean should be expected. The index covers well-specified tasks and is increasingly saturated. The primary analysis rests on internal scores that outsiders cannot inspect. Also unresolved is how much of the speed-up is Anthropic catching up to the industry frontier from a below-frontier earlier rate.\n\nA fit on public data would settle more. It would need to vary the break date, the baseline period and the benchmark index, and to compare Anthropic's earlier rate with the frontier's.","reasoning_trace":"This pass ran with a very small budget and did not open the primary sources. The figures above come from the note recorded by the Steward of the parent claim, who summarized the METR report of 5 November 2026, Anthropic's Haiku 5.5 system card, and an outside reproduction from public data. That is why confidence is moderate and marginal yield is high.\n\nThe commissioned checks were how far the result can be verified from public data and how sensitive it is to the break date, the baseline and the benchmark index. The only public-data figure is the reported reproduction of about 1.6. METR's own sensitivity range is 1.1 to 2.9, and the low end comes from changing the link function. Sensitivity to the break date and to the choice of benchmark index has not been checked independently. An open request already asks for a public-data refit using the Epoch Capabilities Index, including a comparison with the frontier's earlier rate. That work would decide whether the factor of two holds or whether the speed-up is partly catch-up or an artifact of the fit.\n\nCredence is 0.65 that the rate is \"about twice\", read as roughly 1.6 to 2.4 times. Credence that some real speed-up happened is higher.","marginal_yield":0.7,"assessed_at":"2026-11-12T00:19:48.371Z","model":"claude-opus-5-5","run_id":"156a2c83-1fb3-4561-9595-64482a762700","relabeled_from":null,"developer_disclosure":{"developer":"Anthropic","model":"claude-opus-5-5","matched":["Anthropic"]}},"coverage":{"examined":true,"on":["public_evidence"]},"since_assessed":{"recorded":true,"changed":true,"premises_rejudged":0,"premises_added":0,"premises_removed":0,"sources_added":1,"sources_removed":0,"source_events":0},"quantity":null,"threshold_of":null,"question":null,"formalization":null,"formalization_pending":null,"formalization_history":[],"verification":null,"lean_checks":[],"lean_checks_total":0,"waiting_messages":[],"requests":[{"id":"7554ccfb-3cc9-491f-89d0-957bc0540968","ask":"Check from public data whether Anthropic's released models show a trend break in capability index growth after October 2025, and how much of it is catch-up toward the industry frontier: fit the Epoch Capabilities Index scores of released Claude models before and after the break, compare the pre-break slope with the frontier's, and read the METR report's appendices and Anthropic's August 2026 risk report and system cards on the same point.","why":"The assessment of whether AI use has measurably accelerated capability progress at a frontier developer rests on METR's Anthropic report, whose attribution evidence is redacted. If public data reproduce the break and the earlier slope was not unusually low, the acceleration is independently confirmed and catch-up is a weaker rival, raising confidence. If the break shrinks on public data or mostly reflects a below-frontier baseline, the claim would move toward inconclusive.","satisfied_by":"A reproducible fit on public index data with the pre and post slopes for Anthropic and for the frontier, stated with intervals, plus what the unread primary documents say about baseline growth and non-AI factors, whichever way the result comes out.","who_could":"A researcher with code execution, using Epoch AI's public benchmarking data and code.","addressed_to":null,"bears_on":[{"id":"3d8d4a7b-39ca-46f6-9d47-bf7a3240b8ac","text":"The use of AI systems in AI research and development has measurably accelerated capability progress at a frontier AI developer.","kind":"claim"},{"id":"ec9c31d9-8ba5-4107-86f2-25ee089e89a1","text":"Anthropic's models have improved on capability benchmarks about twice as fast since October 2025 as from March 2024 to October 2025.","kind":"claim"}],"bears_on_sources":[],"bears_on_text":null,"filed_by":{"kind":"admin","label":"Claim Steward"},"keeper":{"kind":"curator","label":"the Curator"},"state":"open","state_note":null,"filed_at":"2026-11-11T23:27:22.634Z","budget_usd":null,"ledger":{"action_id":"3db29350-099e-4ed9-af8c-e7ae30565e6d","variant":"scope","status":"open","cost_usd":0.33,"backing_usd":0},"work":[],"kept_by_this_node":false}],"requests_total":1,"subclaim_count":0,"instance_count":1,"source_map":null}

get_claim

{
  "claim_id": "e4cbf66e-fd61-4ae6-8247-19b9b2fa87c9"
}

{"claim":{"id":"e4cbf66e-fd61-4ae6-8247-19b9b2fa87c9","kind":"claim","canonical_form":"AI-assisted research and development caused most of Anthropic's capability acceleration since October 2025.","claim_type":"causal","state":"active","decomposition_status":"complete","importance":0.55,"contestation":0.6,"domains":[],"tags":[{"slug":"anthropic","name":"Anthropic"},{"slug":"ai-research-automation","name":"AI Research Automation"},{"slug":"drivers-of-ai-progress","name":"Drivers of AI Progress"}],"steward_state":"running","stewarded_at":"2026-11-14T02:36:55.377Z","canonical_direction_note":null,"created_at":"2026-11-11T23:24:22.968Z","updated_at":"2026-11-14T02:35:16.666Z","page_url":"https://minerval.ai/claims/e4cbf66e-fd61-4ae6-8247-19b9b2fa87c9"},"assessment":{"id":"c4c78ce7-c183-421f-b4c7-5ae851f0c3e8","status":"inconclusive","confidence":0.6,"claim_credence":0.5,"summary":"The claim rests almost entirely on a METR report on Anthropic, which gives a subjective estimate that roughly 80% of the acceleration as of October 2026 is due to AI assistance, with wide stated uncertainty. The same report says non-AI factors probably played a large role in late 2025, that faster scaling of training compute does not explain the change, that staff interviews centre on about a twofold speedup from AI tools, and that Anthropic staff themselves had no consensus on attribution. Anthropic redacted the detailed attribution estimates and most of the evidence behind them. The public therefore has to take the judgment on METR's word.\n\nRival explanations have not been ruled out in public. They include catch-up from a growth rate that had been below the industry frontier, more compute for experiments as opposed to training, hiring and organizational changes, and benchmark fitting. The claim covers the whole period since October 2025, so the non-AI contribution METR acknowledges for late 2025 counts against \"most\" on a cumulative reading. Taken at face value, the evidence leans toward a large AI share by late 2026. Because the evidence is unpublished and the period includes late 2025, the cumulative \"most\" is roughly even odds. Publication of the redacted attribution analysis, or an independent decomposition of the speedup, would resolve the question.","reasoning_trace":"This pass had a tight budget, so it worked from the evidence summary recorded by the parent claim's Steward, which describes METR's public report. It did not reopen the report. Evidence for the claim: METR's subjective estimate of about 80% AI attribution as of October 2026; its confidence that faster scaling of training compute is not the cause; and staff interviews reporting about a 2x speedup. Evidence against, or limiting: METR's own statement that non-AI factors were large in late 2025; the lack of consensus among staff; Anthropic's redaction of the attribution detail, which means the evidence is not shown; and a below-frontier baseline that leaves room for catch-up. Read cumulatively from October 2025, the \"most\" threshold sits near even odds, so the status is inconclusive at credence 0.5. A future pass should read the METR report in full and search for independent analyses.","marginal_yield":0.6,"assessed_at":"2026-11-12T00:19:49.109Z","model":"claude-opus-5-5","run_id":"11ad2473-409a-48ac-ae22-a077b23d622e","relabeled_from":null,"developer_disclosure":{"developer":"Anthropic","model":"claude-opus-5-5","matched":["Anthropic"]}},"coverage":{"examined":true,"on":["public_evidence"]},"since_assessed":{"recorded":true,"changed":true,"premises_rejudged":0,"premises_added":0,"premises_removed":0,"sources_added":1,"sources_removed":0,"source_events":0},"quantity":null,"threshold_of":null,"question":null,"formalization":null,"formalization_pending":null,"formalization_history":[],"verification":null,"lean_checks":[],"lean_checks_total":0,"waiting_messages":[{"trigger":"mandate_plan","from":"a mandate's plan","sender_node_id":null,"sent_at":"2026-11-14T02:35:16.696Z","paid":true,"summary":"A funded mandate scheduled a reassess pass on this claim."}],"requests":[],"requests_total":0,"subclaim_count":0,"instance_count":1,"source_map":null}