Run step
Step 12: tool results
search_similar_claims
Input
{
"limit": 5,
"query": "OpenAI will have an automated AI researcher by March 2028"
}{
"query": "OpenAI will have an automated AI researcher by March 2028",
"results": [
{
"id": "200278b7-b70d-4549-8ab4-da228093c781",
"kind": "claim",
"text": "OpenAI achieved an automated AI research intern by September 2026.",
"claim_type": "empirical_derived",
"state": "active",
"similarity_score": 0.7985603886646055,
"importance": 0.45,
"assessment_status": null,
"assessment_confidence": null
},
{
"id": "8dc0827d-5e18-4e56-8f0b-9516bbde379b",
"kind": "claim",
"text": "At least one AI lab will fully automate its AI research by the end of 2027.",
"claim_type": "empirical_derived",
"state": "active",
"similarity_score": 0.7510363351228908,
"importance": 0.6,
"assessment_status": null,
"assessment_confidence": null
},
{
"id": "f714e062-8b4c-4bb8-9f3d-38e74f68d06a",
"kind": "claim",
"text": "All major AI labs will have fully automated their AI research by the end of 2027.",
"claim_type": "empirical_derived",
"state": "active",
"similarity_score": 0.7377921060711458,
"importance": 0.3,
"assessment_status": "doubtful",
"assessment_confidence": 0.85
},
{
"id": "44caf44c-3dbc-4084-944f-687f2fb28510",
"kind": "claim",
"text": "Anthropic models available by July 15, 2026, could fully automate AI R&D and R&D in other key domains.",
"claim_type": "empirical_derived",
"state": "active",
"similarity_score": 0.690430125609909,
"importance": 0.6,
"assessment_status": null,
"assessment_confidence": null
},
{
"id": "aa36d48a-c988-4991-a701-50102794ef74",
"kind": "quantity",
"text": "Calendar year in which AI systems first fully automate AI research and development",
"claim_type": "empirical_derived",
"state": "active",
"similarity_score": 0.6735274831809864,
"importance": 0.85,
"assessment_status": null,
"assessment_confidence": null
}
],
"count": 5
}search_similar_claims
Input
{
"limit": 5,
"query": "Frontier AI labs will deliberately pace or slow development as research automation approaches"
}{
"query": "Frontier AI labs will deliberately pace or slow development as research automation approaches",
"results": [
{
"id": "142793f4-48e0-427e-9662-866a34fa2dcb",
"kind": "claim",
"text": "Much of frontier AI progress comes from automatable experimentation whose pace depends on tools and resources.",
"claim_type": "empirical_derived",
"state": "active",
"similarity_score": 0.6880806917775043,
"importance": 0.65,
"assessment_status": "supported",
"assessment_confidence": 0.55
},
{
"id": "f07e018c-7df9-4db6-972c-d7feb111e699",
"kind": "claim",
"text": "AI systems will become capable of fully automating frontier AI research.",
"claim_type": "empirical_derived",
"state": "active",
"similarity_score": 0.6569868152721076,
"importance": 0.75,
"assessment_status": "supported",
"assessment_confidence": 0.55
},
{
"id": "3699eb33-020b-43bd-8da6-469f19575f06",
"kind": "claim",
"text": "Partial automation of AI R&D tasks will substantially speed AI software progress before full automation.",
"claim_type": "causal",
"state": "active",
"similarity_score": 0.6544125492621408,
"importance": 0.7,
"assessment_status": "supported",
"assessment_confidence": 0.65
},
{
"id": "81eeb61b-4ce4-4d0b-891d-11e970d8f669",
"kind": "claim",
"text": "As of 2026, frontier AI agents can perform research engineering but struggle with critical stages of the AI research lifecycle.",
"claim_type": "empirical_derived",
"state": "active",
"similarity_score": 0.6521609453154277,
"importance": 0.6,
"assessment_status": "supported",
"assessment_confidence": 0.7
},
{
"id": "a6037281-5051-4d1f-bd95-5546955a242d",
"kind": "quantity",
"text": "Date when a frontier AI company first has an AI matching top researchers' research taste and fully automating coding",
"claim_type": "empirical_derived",
"state": "active",
"similarity_score": 0.6520640488647647,
"importance": 0.8,
"assessment_status": null,
"assessment_confidence": null
}
],
"count": 5
}search_similar_claims
Input
{
"limit": 6,
"query": "Acceleration measured on benchmarks of well-specified tasks generalises to open-ended research work"
}{
"query": "Acceleration measured on benchmarks of well-specified tasks generalises to open-ended research work",
"results": [
{
"id": "d99ac54d-f549-4449-aa8f-70ab070e16da",
"kind": "claim",
"text": "Time-horizon trends measured on benchmark software tasks generalize to real-world software work.",
"claim_type": "empirical_derived",
"state": "active",
"similarity_score": 0.6236930416450194,
"importance": 0.7,
"assessment_status": "inconclusive",
"assessment_confidence": 0.6
},
{
"id": "ecb10d1f-b391-4e54-b645-8133ad3827ae",
"kind": "claim",
"text": "Recent acceleration measured on well-specified AI benchmarks will not generalize to AI research and development.",
"claim_type": "empirical_derived",
"state": "active",
"similarity_score": 0.6174983103161648,
"importance": 0.6,
"assessment_status": "inconclusive",
"assessment_confidence": 0.6
},
{
"id": "2b2b6f4e-1b6c-4608-b386-865259266ef3",
"kind": "claim",
"text": "Growth in AI task-completion time horizons on benchmarks tracks progress toward AI that can automate AI research or most cognitive work.",
"claim_type": "empirical_derived",
"state": "active",
"similarity_score": 0.5303440094664422,
"importance": 0.6,
"assessment_status": "inconclusive",
"assessment_confidence": 0.6
},
{
"id": "2fba320c-eb0d-4c17-85fe-84047a63b83c",
"kind": "claim",
"text": "AI benchmarks overestimate models' capabilities on real-world tasks.",
"claim_type": "evaluative",
"state": "active",
"similarity_score": 0.5292282529893315,
"importance": 0.6,
"assessment_status": "supported",
"assessment_confidence": 0.7
},
{
"id": "78d6c0a0-a6d8-4b57-89eb-f193be0c0e0e",
"kind": "claim",
"text": "AI agent evaluations built around narrow, verifiable tasks do not assess open-ended research ability.",
"claim_type": "evaluative",
"state": "active",
"similarity_score": 0.5234562343375326,
"importance": 0.4,
"assessment_status": "supported",
"assessment_confidence": 0.7
},
{
"id": "f2bff4f2-c6ee-4950-bf76-eb83969f2707",
"kind": "question",
"text": "How can measured AI productivity effects on developers be reconciled with benchmark scores and anecdotal reports of AI helpfulness and adoption?",
"claim_type": "empirical_derived",
"state": "active",
"similarity_score": 0.4934841766880145,
"importance": 0.55,
"assessment_status": null,
"assessment_confidence": null
}
],
"count": 6
}query_sql
Input
{
"sql": "select id, url, left(title,100) as title, published_date from sources where title ilike '%Import AI 455%' or title ilike '%About To Start Building Themselves%' or url ilike '%import-ai-455%' or title ilike '%automate AI research yet%' or url ilike '%2607.27191%' limit 10"
}{"success":true,"columns":["id","url","title","published_date"],"rows":[["86215cb6-dd1a-40af-963f-2cdba81c2641","https://arxiv.org/html/2607.27191v2","Can AI agents conduct open-ended AI research? Early evidence from two case studies","2026-07-29"],["87406e11-4df5-49b3-8558-11e864316894","https://importai.substack.com/p/import-ai-455-automating-ai-research","Import AI 455: AI systems are about to start building themselves.",null],["424396ec-74d1-4cfb-a175-6847498f5197","https://arxiv.org/abs/2607.27191","Can AI agents conduct open-ended AI research? Early evidence from two case studies","2026-07-29"]],"row_count":3,"truncated":false,"elapsed_ms":138,"plan_cost":121}get_claim
Input
{
"claim_id": "83df12e2-68f5-4d25-8327-28d24ab513be"
}{"claim":{"id":"83df12e2-68f5-4d25-8327-28d24ab513be","kind":"claim","canonical_form":"By the end of 2027, AI systems will be able to complete tasks that take a skilled person weeks.","claim_type":"empirical_derived","state":"active","decomposition_status":"complete","importance":0.6,"contestation":0.55,"domains":[],"tags":[{"slug":"ai-development-forecasting","name":"AI Development Forecasting"},{"slug":"ai-agents","name":"AI Agents"},{"slug":"metr-time-horizon-benchmark","name":"METR Time Horizon Benchmark"}],"steward_state":"done","stewarded_at":"2026-11-14T02:59:37.641Z","canonical_direction_note":"The source advances the positive, conditional forecast that AI could reach weeks-long task capability by 2027, rather than denying it or leaving the matter as a question.","created_at":"2026-11-13T22:12:27.693Z","updated_at":"2026-11-14T03:00:01.296Z","page_url":"https://minerval.ai/claims/83df12e2-68f5-4d25-8327-28d24ab513be"},"assessment":{"id":"4ebf2692-49cb-4661-b2de-2b6ed00f0002","status":"supported","confidence":0.65,"claim_credence":0.75,"summary":"The forecast rests mainly on METR's measurements of AI \"time horizons\": the length of software tasks, measured by how long skilled humans take, that frontier agents complete with 50% success. In early 2026 the leading public models measured about 12 hours (Claude Opus 4.6, after a correction from 14.5), with stronger models put at 16 hours or more, and [[claim:d8be49b1-0f3c-43e5-8426-cc8bc25d9165|recent doubling times]] are estimated at roughly 3.5 to 4.5 months. If that growth continues even at the older seven-month pace, the 50% horizon passes a 40-hour working week during 2027, and at the recent pace it passes several weeks well before the end of the year.\n\nThere is also direct evidence that the threshold is already crossed for some tasks. In April 2026, Epoch AI and METR's MirrorCode benchmark reported Claude Opus 4.6 fully reimplementing a 16,000-line program that engineers estimated would take a human 2 to 17 weeks. Those tasks are precisely specified and checkable, and they have no human baselines, so they show capability on a favourable kind of task, not on weeks-long work in general.\n\nThe main uncertainties are about meaning rather than trend. The measured horizons are for software tasks at 50% reliability; 80% horizons are several times shorter, and messier, less well-specified work in other fields has not been shown to follow the same curve. METR's own suite was nearly saturated in 2026, so a weeks-long horizon may not be cleanly measurable by the end of 2027. Read as \"frontier AI will be able to complete, with substantial reliability, some or typical software tasks that take a skilled person a week or more\", the claim is well supported. Read as reliable performance across weeks-long tasks in general, it is much less certain. The most prominent statement of the forecast comes from Anthropic, whose own models are the subject of the evidence, though the underlying measurements are METR's and Epoch's.","reasoning_trace":"Own model: the claim is a forecast whose truth depends on (a) the trend in task length agents can complete, (b) whether the trend holds 2026-2027, (c) how \"able to complete tasks that take a skilled person weeks\" is read (success rate, domain, task type).\n\nEvidence: METR Time Horizon 1.1 (Jan 2026) put post-2023 doubling time at about 129 to 131 days; METR's May 19, 2026 Frontier Risk Report fit a 105-day doubling to frontier models since 2024 and gave the public frontier around 12 hours (95% CI 5-61) at 50% and about 1.5 hours at 80%, with internal models likely at 16 hours or more (reported in secondary summaries, metavert.io and ai2027-tracker.com; METR's own pages not read in full this pass). Extrapolation: 16 h x 2^(21/4.3) is about 470 h by end of 2027; at a 7-month doubling, about 128 h (around 3 weeks). Both pass one work week (40 h).\n\nDirect evidence: Epoch AI / METR MirrorCode (April 2026, https://epoch.ai/publications/mirrorcode-preliminary-results): Opus 4.6 reimplemented gotree (~16k lines Go), estimated at 2 to 17 human-weeks. METR noted these horizons are far longer than on its standard suite, possibly because the specs are precise and checkable or because labs train on similar tasks.\n\nAgainst or caveats: 50% vs 80% reliability gap; software-only domain; benchmark saturation complicates measurement; external-validity questions (e.g. METR's 2025 developer RCT found AI slowed experienced developers); trend breaks are possible (compute or data limits), though no break is evident through 2026. The claim's source (Anthropic, \"When AI builds itself\") has a commercial interest in its models seeming capable; its trend figures match METR's independent measurements.\n\nCredence 0.75 on the natural reading (by end of 2027, frontier systems complete at 50% success or better at least some meaningful class of tasks that take skilled humans a week or more). Lower on a stricter reading of reliable general performance. I did not search for evidence after mid-2026 beyond noting that METR published an assessment of AI R&D acceleration at Anthropic in October 2026; a later pass could read the latest METR measurements.","marginal_yield":0.35,"assessed_at":"2026-11-14T02:59:54.011Z","model":"claude-opus-5-5","run_id":"65f140c8-244e-48f3-b4d4-66b78cd32912","relabeled_from":null},"coverage":{"examined":true,"on":["public_evidence"]},"since_assessed":{"recorded":true,"changed":false,"premises_rejudged":0,"premises_added":0,"premises_removed":0,"sources_added":0,"sources_removed":0,"source_events":0},"quantity":null,"threshold_of":null,"question":null,"formalization":null,"formalization_pending":null,"formalization_history":[],"verification":null,"lean_checks":[],"lean_checks_total":0,"waiting_messages":[],"requests":[],"requests_total":0,"subclaim_count":1,"instance_count":2,"source_map":null}