Step 8: tool results

provenance_read_source

{
  "offset": 56000,
  "max_chars": 16000,
  "source_id": "a002b1fe-529f-408f-b209-3455cb652171"
}

{"success":true,"source":{"id":"a002b1fe-529f-408f-b209-3455cb652171","url":"https://blog.aifutures.org/p/q35-timelines-update-looks-like-the","title":"Q3.5 Timelines Update: Looks Like the Intelligence Explosion Will Happen in 2027 After All","source_type":"blog_post"},"origin":"stored","total_chars":113075,"offset":56000,"content":"additional speed-up are much more difficult. If so, this data could confirm that prior without necessarily implying that the second batch of capabilities is very close.\n\nHowever, this is just a pretty specific prior, at least if it’s confident about the size and distance between those batches of tasks. If you smooth this out with a greater amount of uncertainty (e.g. about whether the current batch of speed-up has ended yet or will keep accelerating, or a more uniform prior over when different capabilities will appear) then you may get something closer to the above. See also discussion of this in the context of the AI Futures Model above .\n\nAdjustments for what the filtered AIFM could be missing\n\nAs mentioned above, we think the AIFM 39 did a somewhat poor job predicting the developments described in the METR report. For example, it cannot account for a doubling in the pace of progress starting in late 2025 that was initially mostly non-AI-attributed. Therefore, the trajectories filtered from the AIFM had a much slower pace of progress from late 2025 to early 2026 than METR’s report describes.\n\nThe research taste of the AIs in the model trajectories above tends to be higher than we actually expect is true of today’s AIs.\n\nWithin the AIFM, the most common way to reach 1.7x acceleration by Nov 1 is for the AIs to already have strong research taste. So the filtered trajectories have a median research taste of roughly −0.3 to +0.2 SD on Sep 25, 2026 (see the table in “Filtering for a single acceleration level” above) and +0.4 to +0.7 SD on Nov 1. Our best guess for current AIs is the one used in the linear-extrapolation Monte Carlo above: a median of −1.5 SD on Sep 25, 2026. Since the filtered trajectories start from better research taste than we think current AIs have, they probably reach SAR a few months too early.\n\nIn the below table we compare the effects of various ways of compensating for this mismatch:\n\nMeasuring when the model predicts research taste to be 4.6 SDs above its Sep 25, 2026 level (i.e., implicitly assuming that research taste on Sep 25 is actually −1.5 SD, the median used in the linear-extrapolation Monte Carlo above, so that 4.6 additional SDs are required to get to the 99.9th percentile required for SAR)\n\nMeasuring when the model predicts taste to be 5 SDs above the median researcher (2 SD more than usual)\n\nShifting Daniel’s distribution over the AI research taste at AC parameter down by 2 SD.\n\nThe effect size on SAR medians typically appears to be 1–4 months.\n\nThe AIFM doesn’t take into account the discrete nature of training runs, which could overestimate takeoff speeds. In other words, the AIFM assumes that a new innovation will be instantly applied to models and then cause further speed-ups in the rate of innovation discovery. But in practice, it might take months for pretraining improvements to show up in a fully trained model, and perhaps weeks for RL training improvements to show up. Our rough, intuition-based best guess is that this pushes back the SAR median by about 1–3 months relative to the AIFM’s predictions. However, this concern does not apply to the model based on linear extrapolation, since it does not rely on further speed-ups from here.\n\nOverall forecasts\n\nForecast table\n\nThese forecasts are conditional on people not “pacing the frontier” (i.e., delaying when a given capability level is achieved) any more in the future than in the recent past.\n\nEli’s forecast\n\nThe simple research taste extrapolation feels fairly compelling to me. As a reminder, it results in a Monte Carlo median of Jul 2027 and a simple plug-in value of Sep 2027. Unfortunately, we don’t have great evidence on either of its key parameters. Remember, this method is conservative in that it doesn’t account for future pace of progress increases due to automation.\n\nMeanwhile, regarding the filtered AIFM: The TED-AI median with a 1.5x larger SAR→TED-AI gap is Sep 2027 when only conditioning on 1.7x in Nov, and Nov 2027 when also conditioning on 1.2x in May. More broadly, the fact that it seems like acceleration has been going more gradually than we expected updates me toward somewhat slower takeoff.\n\nI ran a Monte Carlo using the AIFM over Nov 2026 acceleration levels of 1 + 0.6 [0.06, 6], 40 clipped at 7 (i.e., trajectories are filtered from AIFM with weight according to this distribution).\n\nI set my median slightly lower than the 1.7x AI-attributed acceleration, given that I had a strong prior against these results. However, I feel compelled to update nearly all the way to their result given that I respect the people who did the assessment, and as mentioned above, the results don’t appear to have confirmed their preexisting beliefs.\n\nThis resulted in a SAR median of March 2027, and a TED-AI median of November 2027 (February 2028 with a 1.5x larger SAR→TED-AI gap to take into account increased jaggedness). This Nov 2027 TED-AI median is about 4 months later than when conditioning on exactly 1.7x (Jul 2027), mostly because of the added uncertainty: lower acceleration levels delay TED-AI more than higher ones speed it up. 41 I didn’t run a Monte Carlo that included filtering for non-negligible May acceleration, but presumably it would have given results a few months later (as did the filtering without uncertainty).\n\nTo test robustness: with a lower distribution of 1 + 0.4 [0.04, 4] clipped at 5, SAR is about 4 months later (July 2027) and TED-AI 5–6 months later (April 2028, or August 2028 with the larger gap). 42\n\nAdjusting for the fact that the filtered AIFM likely estimates too high current taste, and that it doesn’t model discrete training runs, should also push these forecasts back a bit.\n\nRegarding why the AIFM was wrong : I don’t feel like I have a great sense yet for how to update based on this. Perhaps further analysis will be more revealing. Our inability to account for automation of data generation as a cause of acceleration makes me a bit nervous, and I update a little toward faster takeoff due to us maybe modeling diminishing returns too harshly.\n\nBased just on the models and considerations discussed thus far, I’m at a SAR median of July 2027, and a TED-AI median of August 2028.\n\nSome factors that are pushing me toward even shorter timelines than this:\n\nIndustry rumors\n\nMe being consistently too bearish on AI progress in the past, from my current perspective (see below )\n\nThe forecasting method of “When did this happen in AI 2027?”\n\nSome factors that are pushing me toward marginally longer timelines:\n\nA nagging feeling that the update feels so large without being based on commensurate evidence (this is related to how the METR report had most of the relevant evidence redacted.)\n\nMy experiences with Manifold when I’ve bet a ton of mana before taking the time to think properly about whether I’m so confident, are calling to me.\n\nAll of our methods make an assumption about “research taste” being the other key skill involved in AI development besides coding. Maybe things are more complicated and there are key aspects of taste that lag way behind others, or there are other key skills in AI development that will lag behind research taste (e.g., running large projects for long periods of time).\n\nI don’t think these sorts of intuitions have performed that well for me in the past, though. They probably would have predicted that math would be harder to automate than it actually was, for example.\n\nThere’s a voice inside my head saying “Surely this crazy stuff isn’t actually happening while most of the world is barely paying any attention, right?? Where is the smart money?” See e.g., the markets not reacting at all to the METR report while in my view it’s probably the biggest single piece of evidence ever on near-term economic growth.\n\nMore generally, it feels like there are not many people even more bullish than the numbers I laid out above, and there are many people who are more bearish.\n\nThat said, this sort of instinct has performed very poorly for me in the past.\n\nAlso, the people I defer to most are probably many of the co-authors of this blog post and they are making similar predictions. Perhaps I should be deferring more to Daniel, given his track record. (Also people like Ajeya, who wrote that the METR report dramatically shortened her timelines . And notably, Sentinel, a group of forecasters who have historically predicted slower AI progress than me, now predicts a median of early 2028 for superintelligence . 43 )\n\nThere are probably some emotional factors where I don’t like admitting that I’ve been wrong, which I’m trying to do my best to set aside.\n\nOverall, I’ll adjust longer but only by a bit. I’m at a SAR median of August 2027, and a TED-AI median of September 2028.\n\nBrendan’s forecast\n\nFor SAR , I am mostly relying on AECI extrapolation, with some adjustment based on serial time required for new training runs. I don’t trust the filtered AIFM to provide very much additional value.\n\nHere are my current intuitive credences in various possible amounts of 2024-2025 progress in 2027, measured using some idealized version of AECI, as usual conditional on no regulation/significant voluntary slowdown:\n\nAt least as much progress as seen in 2026: 90%\n\nI see various reasons to expect faster progress, such as increasing AI R&D automation\n\nI think the underlying drivers of progress (leading indicators) are unlikely to have slowed (for one, I think METR would have mentioned this if they had). The lag from research to capabilities-of-internally-deployed-models is probably at least 3 to 5 months for new base models, so I’m very confident extrapolating through the first half of 2027.\n\nI don’t really see any compelling reasons to expect slower progress overall. Even if we aren’t at self-sustaining acceleration in mid-2027, extrapolating a yearlong trend feels safe, and so far we’ve only ever seen such trends speed up rather than slow down.\n\n>2x the progress seen in 2026 (roughly 4 years of 2025 progress): 60%\n\n>4x the progress seen in 2026 (roughly 8 years of 2025 progress): 40%\n\n>8x the progress seen in 2026 (roughly 16 years of 2025 progress): 20%\n\nFor TED-AI , I am combining\n\nAn update towards larger distance in ECI space between SAR and TED-AI (since SAR being near implies greater jaggedness) with\n\nAn update towards faster, more explosive takeoff, based on the unexplained non-AI-derived speedup METR describes.\n\nThis likely indicates an acceleration in software progress that the AIFM (and most Jones-style models of software R&D) would not have predicted. This is the second time this has happened, the first being reasoning models .\n\nDaniel’s forecast\n\nAbout a year ago, I wrote :\n\nObviously, if benchmark trends (especially horizon length) keep going at the current pace or accelerate, that’ll be an update towards shorter timelines. Right now I still think it’s more likely than not that there’ll be a slowdown in the next year or two.\n\nI’m eager to get more information about coding uplift. When we have a reliable trend of coding uplift to extrapolate, I’ll at the very least want to redo my estimates of the model parameters to fit that coding uplift trend, and possibly I’d want to rethink the model more generally to center on coding uplift instead of on horizon length.\n\nIf AI revenue growth stays strong (e.g. 4xing or more in 2026) that’s evidence for shorter timelines vs. if it only grows 2x or less that’s evidence for longer timelines.\n\nWelp. Seems like benchmark trends (AECI, even ECI) have not only kept pace, they have accelerated. And it seems like coding uplift is really huge right now, unclear how huge. As for revenue, it seems to have gone up about 6x , despite the leading AI company spending ~70% of its compute on R&D.\n\nIn the Q2.5 timelines update written mid-August , I said my 50% mark for AC was December 2027 and my 50% mark for SAR was December 2028. Since then various pieces of evidence have appeared and weighed on my mind, including industry insider rumors, public statements by AI companies, the mathopocalypse, and even the Hugging Face incident which was an example of AIs doing research projects end-to-end. The biggest single piece of evidence, however, is this METR report. I still don’t trust it completely, of course, and so I still have substantial credence that things will turn out to take longer than the models and trendlines naively imply for one reason or another. But on the other hand, it’s also scarily plausible that things will go even faster than the models or trendlines naively imply, for several reasons. For example, humans in general might have a bias towards underestimating AI capabilities progress – this certainly seems true outside the AI industry, but even within the AI industry it seems more have underestimated it than overestimated it historically. Unless this bias somehow doesn’t apply to Anthropic or METR today, then, it seems we should try to correct for it, but that would mean SAR is happening this winter!?!? At any rate, mashing together all these factors, it seems to me that I should roughly go with what the naive extrapolations and models say – so, let’s say May 2027 for SAR median, i.e., 50% chance of SAR by May 2027.\n\nLukas’s forecast\n\nHere are some things I believe:\n\nSignificant probability on SAR early 2027.\n\nThe METR report flags that they have less evidence about current acceleration levels than past acceleration levels, and that some redacted evidence suggested the possibility of hyperbolic acceleration and underestimating the current pace.\n\nI was surprised by the pace of progress being this fast right now and should apparently be widening my error bars for aggressive movement soon.\n\nProgress can be lumpy as illustrated by Mythos.\n\nMedian timing of SAR sometime 2027.\n\nMost models that aren’t clearly too conservative (such as not accounting for automation) predict SAR in 2027 and I don’t want to go too much later than them.\n\nIt seems like, if I had adopted the views of various people who have ~2027 timelines, I would have been better at predicting recent events.\n\nSignificant probability on SAR not happening in 2027.\n\nThe situation is confusing. My prior was that automation in 2027 wasn’t going to happen, and the evidence is not robust enough to fully shatter that.\n\nThe models still have weaknesses relative to humans, and it’s possible that fixing some of them might be necessary for SAR and might see less fast improvements than the models’ strengths. Especially on hard-to-measure tasks, where it’s harder to train models. Unfortunately, it’s tricky to measure whether models are or aren’t improving as fast on these tasks.\n\nIt’s plausible that progress is fast and accelerating but we just won’t quite make it to SAR next year.\n\nProbability mass early 2027 vs. late 2027 doesn’t seem wildly different to me, but late seems a bit more likely than early.\n\nI give some weight to the research taste extrapolations which are really surprisingly fast. But I’m a bit suspicious that I don’t really know what research taste is or how to reason about it.\n\nI do quite like the model which is Extrapolating “additional software R&D uplift per additional unit of ECI” based on the self-reported serial labor uplift, which predicts SAR late 2027. It seems like a good combination of grounded in data and not relying on concepts like “research taste” that I don’t quite know what they mean. My main worry is it’s a bit too conservative relative to assuming that AI R&D uplift has been more recent, which implies SAR early summer instead.\n\nOverall, I think my median should be sometime late-ish next year, to account for the probability mass going into future years as well.\n\nThen about later milestones, like top-human-expert-dominating AI:\n\nConditional on SAR next year, accelerating growth seems likely.\n\nIt would have backed up the idea that we have accelerating growth this year.\n\nSo probably it happens soon after. Because progress is probably really fast after SAR.\n\nThe median of TED-AI will ","truncated":true,"next_offset":72000,"note":"More text follows; call again with offset 72000 to continue."}