AI agents’ 50%-reliability software-task horizon will continue doubling approximately every seven months.
3 events · 1 assessment · 1 decision
Structured and assessed
First pass. Linked the existing quantities for the 2019-2025 doubling time and the post-2024 doubling time, the existing claim about METR TH1.1 being unreliable above 16 hours (assumes), and the existing claim that compute enables complex tasks. Minted one new subclaim: frontier compute growth will slow before 2030. Created three arguments. Assessed doubtful (credence 0.2), because the TH1.1 data show about 4-month doubling since 2024, faster than the seven months stated. The parent (month-long tasks by 2030) relies on the looser 'at least this fast' reading, which remains supported; I flagged this in the assessment. Notifying the parent.
Assessed Doubtful
verdict confidence 0.65 · credence 0.20
The seven-month figure comes from METR's March 2025 study, which found that the length of software tasks frontier AI agents complete with 50% reliability doubled about every seven months from 2019 to early 2025. As a description of that period the figure is sound. As a forecast that the pace will continue at about seven months, it has not held so far. METR's January 2026 revision (Time Horizon 1.1) puts the doubling time for models since 2023 or 2024 at about four months. Measured horizons bear this out: about one hour for Claude 3.7 Sonnet (February 2025), about five hours for Claude Opus 4.5 (November 2025), about twelve hours for Claude Opus 4.6 (February 2026), and at least sixteen hours for an early Claude Mythos Preview (spring 2026). A seven-month pace would have predicted only about four hours by spring 2026. The claim therefore fails as stated, and it fails in the opposite direction from what skeptics expected. Read loosely, as "doubling at least every seven months", it is so far supported. The parent forecast about month-long tasks by 2030 rests on that looser reading. Whether any steady pace continues is still open. On one side, scaling of compute and reinforcement learning continues. On the other, the growth of training compute may slow before 2030, and some analysts expect the post-2024 burst to fall back toward a slower pace once reinforcement learning is a mature share of spending. A practical limit also matters: METR's current task suite cannot reliably measure horizons above about sixteen hours. Continuation can only be checked on new suites, and their numbers may not compare directly with the old series. A replacement suite that tracks a stable doubling rate through 2027 would settle which pace, if any, holds.
Claim entered the graph