Step 17: assistant

Calls write_argument

{
  "content": "The claim is about every major lab, so it is false if any one of them has not fully automated its AI research by the end of 2027. The labs are not in the same position, contrary to the premise that [[claim:5134390a-7579-4ff3-a177-a91292b300a7|their frontier capabilities are only months apart]]. OpenAI's own target for an automated AI researcher is March 2028. Google DeepMind's October 2026 safety report found no significant acceleration of its AI R&D, with its best model solving about half of internal research engineering tasks. Meta has said it will put most of its compute into serving users instead of racing toward recursive self-improvement, and the leading Chinese developers trail the frontier. The chance that all of them cross a strict line by one date is far smaller than the chance that the first one does.",
  "argument_id": "43781ba1-5e33-40b4-9399-8290889b822b"
}

Calls update_claim_assessment

{
  "status": "doubtful",
  "claim_id": "f714e062-8b4c-4bb8-9f3d-38e74f68d06a",
  "assessment": "This is a forecast with two demanding parts: that AI research becomes fully automated, with AI systems doing the work human researchers now do, and that this happens at every major lab by 31 December 2027. The evidence available in November 2026 makes the first part a live possibility at the leading lab and makes the universal version very unlikely.\n\nThe case for it rests on recent signs of acceleration. In November 2026 METR estimated that Anthropic's AI progress was running at about twice its earlier rate and attributed most of the speed-up to AI, though Anthropic redacted most of the evidence METR found informative. On that basis the AI Futures Project moved its median dates for a superhuman AI researcher at the leading company to between May and November 2027, assuming no deliberate slowdown. If that happened and the rest of the industry were only months behind, the forecast could come true.\n\nMost of what is known points the other way. Whether [[claim:8dc0827d-5e18-4e56-8f0b-9516bbde379b|even one lab fully automates its research by the end of 2027]] is itself more likely false than true: Anthropic's Jack Clark put about 30% on AI R&D with no human involvement by that date, Anthropic reports that no measured part of its R&D is fully autonomous and that humans still supply most research judgment, and AI agents given open-ended research questions have not yet produced work that expert authors accept. The universal step is weaker still. OpenAI's own target for an automated AI researcher is March 2028, after the deadline. Google DeepMind's October 2026 safety report found no significant acceleration of its AI R&D. Meta has said it will prioritise serving users over racing toward recursive self-improvement. The premise that [[claim:5134390a-7579-4ff3-a177-a91292b300a7|the major labs are only months apart]] is doubtful once Meta, xAI and the leading Chinese developers are counted. And the heads of Anthropic, OpenAI and xAI endorsed deliberately pacing frontier development in September 2026, which bears on whether a lab that could remove humans from its research would do so at once.\n\nThe estimate here is roughly a one in four chance that the leading lab has the capability by the end of 2027, about half that for a lab actually running its research without human researchers, and a further factor of about one in eight for every other major lab doing the same by the same date: about 2% overall. A weaker proposition, that every major lab will be running some research projects end to end with AI agents by then, is much more likely and is not what this claim states. The forecast resolves at the end of 2027; a credible report before then that Google DeepMind, OpenAI and Meta had each handed their research to AI systems would overturn this assessment.",
  "confidence": 0.85,
  "claim_credence": 0.02,
  "marginal_yield": 0.2,
  "reasoning_trace": "Prior, formed before reading sources: a universal claim over five or more organisations, with a strict threshold (\"fully automated\") and a deadline about 13 months away, starts low. Each of three things must hold: some lab's AI can do all AI research work by the deadline; labs with that capability actually run research without human researchers; and every major lab, not only the leader, does so by the same date.\n\nSearch (web, 13 November 2026): queries on OpenAI's automated research intern and researcher targets, AI Futures Project timeline updates, METR's assessment of Anthropic, Google DeepMind's position, Jack Clark's forecast, sceptical views (Narayanan and Kapoor), Meta, xAI and Chinese labs, the September 2026 pacing proposal, and the exact phrasing of this claim. No source was found asserting the universal form of the claim; the search for one was a handful of phrase queries and would not have found a statement made in a talk or a paywalled piece. The AI Futures post and the Forecasting Research Institute summary were read directly; most other items were read through search-result extracts and news coverage, not the primary documents.\n\nWhere the evidence moved the prior upward for the first lab:\n- METR, Independent Assessment of AI R&D Acceleration at Anthropic in October 2026 (https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/): about 2x acceleration relative to the earlier trend, roughly 80% attributed to AI. METR states Anthropic redacted a majority of the evidence it found most informative, so the attribution is evidence not shown, examined by a third party that Anthropic admitted and whose report Anthropic could redact.\n- AI Futures Project, Q3.5 Timelines Update, 10 November 2026 (https://blog.aifutures.org/p/q35-timelines-update-looks-like-the): all authors now expect a superhuman AI researcher \"at some point in 2027\" conditional on no substantial deliberate slowdown; individual medians reported between May and November 2027. The authors say their model assigned under 4% to the acceleration they are now conditioning on, that the update rests heavily on the redacted METR report, and that their several extrapolation methods share inputs. Their milestone is a capability at the leading company, not practice at all labs.\n- Jack Clark, Import AI 455 (May 2026), as reported: about 30% for no-human-involved AI R&D by end of 2027, 60%+ by end of 2028. Predates the METR report.\n- Elon Musk (March 2026, as reported) said xAI's full automation would come \"no later than 2027\"; a founder's forecast about his own firm with a record of optimistic dates.\n\nAgainst:\n- OpenAI declared its \"automated research intern\" goal met in September 2026 and restated March 2028 as the target for a full automated AI researcher. A major lab's own stated goal falls after the deadline.\n- Google DeepMind's safety report for Gemini 4 Argon (October 2026, via The Decoder, TNW, Decrypt coverage): \"no significant end-to-end acceleration of AI R&D, AI-driven or otherwise\"; the model solved about half of internal research engineering tasks against a 90% red-flag threshold; 88% of code changes still need human oversight.\n- Forecasting Research Institute, How Close Is Recursive Self-Improvement? (10 November 2026, https://forecastingresearch.substack.com/p/how-close-is-recursive-self-improvement): no measured part of Anthropic's AI R&D is fully autonomous; \"large performance gaps persist\" in Claude's judgement in choosing goals; in the CRUX 2 evaluation the original authors rejected all four AI-written papers; public capability indices show no clear acceleration; forecasts of full automation range from 2027 to 2052; an LLM ensemble gave about 40% to a doubled pace of public progress in any year by 2030.\n- Meta (Zuckerberg, September 2026): committing most compute to serving people \"rather than racing towards recursive self-improvement\".\n- \"We Must Pace the Frontier\" (Amodei, 12 September 2026), endorsed the same day by Altman and Musk, proposes pacing and an eventual speed limit on recursive self-improvement. The AI Futures forecast is explicitly conditional on no such slowdown.\n- Compute bottlenecks (Narayanan; Pachocki's \"constant compute crunch\") limit how far automated researchers substitute for the whole research process.\n\nCombination. P(leading lab has AI able to do all AI research work by end of 2027) about 0.25 to 0.30, between Clark's 30% (before the METR report) and the AI Futures authors' better-than-even, discounted because their update rests on undisclosed evidence from one company, DeepMind reports no acceleration, and open-ended research performance remains poor. P(that lab's research is in fact fully automated by the deadline, given the capability) about 0.5. P(every other major lab also does so by the deadline, given one does) about 0.1 to 0.15: OpenAI's plan would need to run ahead of schedule, DeepMind would need to go from no measurable acceleration to full automation in a year, and Meta, xAI and Chinese developers would need to match. Product about 0.015 to 0.02. Recorded credence 0.02.\n\nStatus is doubtful and not contradicted because the claim is about the future and the strongest recent evidence (the METR report) is moving in the claim's direction, with its key support unpublished. What would change the verdict: publication of the redacted METR evidence showing hyperbolic acceleration, comparable acceleration reports from DeepMind and OpenAI, OpenAI moving its researcher target into 2027, or evidence that the gap between the leader and other labs is closing.\n\nScope note. \"Major AI labs\" is taken to include at least OpenAI, Anthropic and Google DeepMind, and on ordinary usage Meta and xAI; adding Chinese developers lowers the probability further. The claim is read as full automation of each lab's AI research, not as each lab running some automated research projects."
}