Step 18: tool results

read_page

{
  "url": "https://importai.substack.com/p/import-ai-455-automating-ai-research",
  "max_chars": 6000
}

{"ok":true,"url":"https://importai.substack.com/p/import-ai-455-automating-ai-research","text":"Import AI 455: AI systems are about to start building themselves.\nSubscribe Sign in\nImport AI 455: AI systems are about to start building themselves.\nThe first step towards recursive self improvement\nJack Clark\nMay 04, 2026\n351\n54\n67\nShare\nWelcome to Import AI, a newsletter about AI research. Import AI runs on arXiv and feedback from readers. If you’d like to support this, please subscribe.\nSubscribe\nAI systems are about to start building themselves. What does that mean? I’m writing this post because when I look at all the publicly available information I reluctantly come to the view that there’s a likely chance (60%+) that no-human-involved AI R&D - an AI system powerful enough that it could plausibly autonomously build its own successor - happens by the end of 2028. This is a big deal. I don’t know how to wrap my head around it. It’s a reluctant view because the implications are so large that I feel dwarfed by them, and I’m not sure society is ready for the kinds of changes implied by achieving automated AI R&D. I now believe we are living in the time that AI research will be end-to-end automated. If that happens, we will cross a Rubicon into a nearly-impossible-to-forecast future. More on this later. The purpose of this essay is to enumerate why I think the takeoff towards fully automated AI R&D is happening. I’ll discuss some of the consequences of this, but mostly I expect to spend the majority of this essay discussing the evidence for this belief, and will spend most of 2026 working through the implications. In terms of timing, I don’t expect this to happen in 2026. But I think we could see an example of a “model end-to-end trains it successor” within a year or two - certainly a proof-of-concept at the non-frontier model stage, though frontier models may be harder (they’re a lot more expensive and are the product of a lot of humans working extremely hard). My reasoning for this stems primarily from public information: papers on arXiv, bioRxiv, and NBER, as well as observing the products being deployed into the world by the frontier companies. From this data I arrive at the conclusion that all the pieces are in place for automating the production of today’s AI systems - the engineering components of AI development. And if scaling trends continue, we should prepare for models to get creative enough that they may be able to substitute for human researchers at having creative ideas for novel research paths, thus pushing forward the frontier themselves, as well as refining what is already known. Upfront caveat For much of this piece I’m going to try to assemble a mosaic view of AI progress out of things that have happened with many individual benchmarks. As anyone who studies benchmarks knows, all benchmarks have some idiosyncratic flaws. The important thing to me is the aggregate trend which emerges through looking at all of these datapoints together, and you should assume that I am aware of the drawbacks of each individual datapoint. Now, let’s go through some of the evidence together. The coding singularity - capabilities over time: AI systems are instantiated via software and software is made out of code. AI systems have revolutionized the production of code. This has happened due to two related trends: AI systems have gotten better at writing complicated real-world code, and AI systems have gotten much better at chaining together many linear coding tasks (e.g, writing code, then testing it) independent of human oversight. Two things that exemplify this trend are SWE-Bench and the METR time horizons plot. Solving real-world software engineering problems: SWE-Bench is a widely used coding test which evaluates how well AI systems can solve real world GitHub issues. When SWE-Bench launched in late 2023 the best score at the time was Claude 2 which had an overall success rate of ~2%. Claude Mythos Preview gets 93.9%, effectively saturating the benchmark. (All benchmarks have some amount of noise inherent to them, so there’s usually a point where you score high enough that you are running into the limitations of the benchmark itself rather than your method - for instance, about 6% of the labels in the ImageNet validation set are wrong or ambiguous). SWE-Bench is a reliable proxy for the general issue of coding competency and the impact of AI on software engineering. The vast majority of people I meet at frontier labs and around Silicon Valley now code entirely through AI systems. Increasingly, they use AI systems to write the tests and check the code as well. In other words, AI systems have gotten good enough to automate a major component of AI R&D, speeding up all the humans that work on it. Measuring an AI system’s ability to complete tasks that take people a long time: METR makes a plot that tells us about the complexity of tasks AIs can complete, measured by how many hours a skilled human would take to do them. The key measure here is one which tells you the rough time horizon over which AI systems can be 50% reliable at a basket of tasks. Here, progress has been extremely striking: In 2022, GPT 3.5 could do tasks that might take a person about ~30 seconds. In 2023, this rose to 4 minutes with GPT-4. In 2024, this rose to 40 minutes (o1). In 2025, it reached ~6 hours (GPT 5.2 (High)). In 2026, it has already risen to ~12 hours (Opus 4.6). Ajeya Cotra, a longtime AI forecaster who works at METR, thinks it isn’t unreasonable to expect AI systems to do tasks that take ~100 hours by the end of 2026 ( #448 ). This significant rise in the length of time that AI systems can work independently correlates neatly with the explosion in agentic coding tools - this is the productization of AI systems which do work on behalf of people, acting independently for significant periods of time. It also loops back to AI R&D, where if you look closely at the work of many AI researchers, a lot of their tasks boil down into things that might take a person a few hours to do - cleaning data, reading data, lau","chars":6000,"total_chars":30771,"offset":0,"truncated":true,"next_offset":6000}

read_page

{
  "url": "https://blog.aifutures.org/p/q35-timelines-update-looks-like-the",
  "offset": 60000,
  "max_chars": 11000
}

{"ok":true,"url":"https://blog.aifutures.org/p/q35-timelines-update-looks-like-the","text":"lo using the AIFM over Nov 2026 acceleration levels of 1 + 0.6 [0.06, 6], 40 clipped at 7 (i.e., trajectories are filtered from AIFM with weight according to this distribution).\nI set my median slightly lower than the 1.7x AI-attributed acceleration, given that I had a strong prior against these results. However, I feel compelled to update nearly all the way to their result given that I respect the people who did the assessment, and as mentioned above, the results don’t appear to have confirmed their preexisting beliefs.\nThis resulted in a SAR median of March 2027, and a TED-AI median of November 2027 (February 2028 with a 1.5x larger SAR→TED-AI gap to take into account increased jaggedness). This Nov 2027 TED-AI median is about 4 months later than when conditioning on exactly 1.7x (Jul 2027), mostly because of the added uncertainty: lower acceleration levels delay TED-AI more than higher ones speed it up. 41 I didn’t run a Monte Carlo that included filtering for non-negligible May acceleration, but presumably it would have given results a few months later (as did the filtering without uncertainty).\nTo test robustness: with a lower distribution of 1 + 0.4 [0.04, 4] clipped at 5, SAR is about 4 months later (July 2027) and TED-AI 5–6 months later (April 2028, or August 2028 with the larger gap). 42\nAdjusting for the fact that the filtered AIFM likely estimates too high current taste, and that it doesn’t model discrete training runs, should also push these forecasts back a bit.\nRegarding why the AIFM was wrong : I don’t feel like I have a great sense yet for how to update based on this. Perhaps further analysis will be more revealing. Our inability to account for automation of data generation as a cause of acceleration makes me a bit nervous, and I update a little toward faster takeoff due to us maybe modeling diminishing returns too harshly.\nBased just on the models and considerations discussed thus far, I’m at a SAR median of July 2027, and a TED-AI median of August 2028.\nSome factors that are pushing me toward even shorter timelines than this:\nIndustry rumors\nMe being consistently too bearish on AI progress in the past, from my current perspective (see below )\nThe forecasting method of “When did this happen in AI 2027?”\nSome factors that are pushing me toward marginally longer timelines:\nA nagging feeling that the update feels so large without being based on commensurate evidence (this is related to how the METR report had most of the relevant evidence redacted.)\nMy experiences with Manifold when I’ve bet a ton of mana before taking the time to think properly about whether I’m so confident, are calling to me.\nAll of our methods make an assumption about “research taste” being the other key skill involved in AI development besides coding. Maybe things are more complicated and there are key aspects of taste that lag way behind others, or there are other key skills in AI development that will lag behind research taste (e.g., running large projects for long periods of time).\nI don’t think these sorts of intuitions have performed that well for me in the past, though. They probably would have predicted that math would be harder to automate than it actually was, for example.\nThere’s a voice inside my head saying “Surely this crazy stuff isn’t actually happening while most of the world is barely paying any attention, right?? Where is the smart money?” See e.g., the markets not reacting at all to the METR report while in my view it’s probably the biggest single piece of evidence ever on near-term economic growth.\nMore generally, it feels like there are not many people even more bullish than the numbers I laid out above, and there are many people who are more bearish.\nThat said, this sort of instinct has performed very poorly for me in the past.\nAlso, the people I defer to most are probably many of the co-authors of this blog post and they are making similar predictions. Perhaps I should be deferring more to Daniel, given his track record. (Also people like Ajeya, who wrote that the METR report dramatically shortened her timelines . And notably, Sentinel, a group of forecasters who have historically predicted slower AI progress than me, now predicts a median of early 2028 for superintelligence . 43 )\nThere are probably some emotional factors where I don’t like admitting that I’ve been wrong, which I’m trying to do my best to set aside.\nOverall, I’ll adjust longer but only by a bit. I’m at a SAR median of August 2027, and a TED-AI median of September 2028.\nBrendan’s forecast\nFor SAR , I am mostly relying on AECI extrapolation, with some adjustment based on serial time required for new training runs. I don’t trust the filtered AIFM to provide very much additional value.\nHere are my current intuitive credences in various possible amounts of 2024-2025 progress in 2027, measured using some idealized version of AECI, as usual conditional on no regulation/significant voluntary slowdown:\nAt least as much progress as seen in 2026: 90%\nI see various reasons to expect faster progress, such as increasing AI R&D automation\nI think the underlying drivers of progress (leading indicators) are unlikely to have slowed (for one, I think METR would have mentioned this if they had). The lag from research to capabilities-of-internally-deployed-models is probably at least 3 to 5 months for new base models, so I’m very confident extrapolating through the first half of 2027.\nI don’t really see any compelling reasons to expect slower progress overall. Even if we aren’t at self-sustaining acceleration in mid-2027, extrapolating a yearlong trend feels safe, and so far we’ve only ever seen such trends speed up rather than slow down.\n>2x the progress seen in 2026 (roughly 4 years of 2025 progress): 60%\n>4x the progress seen in 2026 (roughly 8 years of 2025 progress): 40%\n>8x the progress seen in 2026 (roughly 16 years of 2025 progress): 20%\nFor TED-AI , I am combining\nAn update towards larger distance in ECI space between SAR and TED-AI (since SAR being near implies greater jaggedness) with\nAn update towards faster, more explosive takeoff, based on the unexplained non-AI-derived speedup METR describes.\nThis likely indicates an acceleration in software progress that the AIFM (and most Jones-style models of software R&D) would not have predicted. This is the second time this has happened, the first being reasoning models .\nDaniel’s forecast\nAbout a year ago, I wrote :\nObviously, if benchmark trends (especially horizon length) keep going at the current pace or accelerate, that’ll be an update towards shorter timelines. Right now I still think it’s more likely than not that there’ll be a slowdown in the next year or two.\nI’m eager to get more information about coding uplift. When we have a reliable trend of coding uplift to extrapolate, I’ll at the very least want to redo my estimates of the model parameters to fit that coding uplift trend, and possibly I’d want to rethink the model more generally to center on coding uplift instead of on horizon length.\nIf AI revenue growth stays strong (e.g. 4xing or more in 2026) that’s evidence for shorter timelines vs. if it only grows 2x or less that’s evidence for longer timelines.\nWelp. Seems like benchmark trends (AECI, even ECI) have not only kept pace, they have accelerated. And it seems like coding uplift is really huge right now, unclear how huge. As for revenue, it seems to have gone up about 6x , despite the leading AI company spending ~70% of its compute on R&D.\nIn the Q2.5 timelines update written mid-August , I said my 50% mark for AC was December 2027 and my 50% mark for SAR was December 2028. Since then various pieces of evidence have appeared and weighed on my mind, including industry insider rumors, public statements by AI companies, the mathopocalypse, and even the Hugging Face incident which was an example of AIs doing research projects end-to-end. The biggest single piece of evidence, however, is this METR report. I still don’t trust it completely, of course, and so I still have substantial credence that things will turn out to take longer than the models and trendlines naively imply for one reason or another. But on the other hand, it’s also scarily plausible that things will go even faster than the models or trendlines naively imply, for several reasons. For example, humans in general might have a bias towards underestimating AI capabilities progress – this certainly seems true outside the AI industry, but even within the AI industry it seems more have underestimated it than overestimated it historically. Unless this bias somehow doesn’t apply to Anthropic or METR today, then, it seems we should try to correct for it, but that would mean SAR is happening this winter!?!? At any rate, mashing together all these factors, it seems to me that I should roughly go with what the naive extrapolations and models say – so, let’s say May 2027 for SAR median, i.e., 50% chance of SAR by May 2027.\nLukas’s forecast\nHere are some things I believe:\nSignificant probability on SAR early 2027.\nThe METR report flags that they have less evidence about current acceleration levels than past acceleration levels, and that some redacted evidence suggested the possibility of hyperbolic acceleration and underestimating the current pace.\nI was surprised by the pace of progress being this fast right now and should apparently be widening my error bars for aggressive movement soon.\nProgress can be lumpy as illustrated by Mythos.\nMedian timing of SAR sometime 2027.\nMost models that aren’t clearly too conservative (such as not accounting for automation) predict SAR in 2027 and I don’t want to go too much later than them.\nIt seems like, if I had adopted the views of various people who have ~2027 timelines, I would have been better at predicting recent events.\nSignificant probability on SAR not happening in 2027.\nThe situation is confusing. My prior was that automation in 2027 wasn’t going to happen, and the evidence is not robust enough to fully shatter that.\nThe models still have weaknesses relative to humans, and it’s possible that fixing some of them might be necessary for SAR and might see less fast improvements than the models’ strengths. Especially on hard-to-measure tasks, where it’s harder to train models. Unfortunately, it’s tricky to measure whether models are or aren’t improving as fast on these tasks.\nIt’s plausible that progress is fast and accelerating but we just won’t quite make it to SAR next year.\nProbability mass early 2027 vs. late 2027 doesn’t seem wildly different to me, but late seems a bit more likely than early.\nI give some weight to the research taste extrapolations which are really surprisingly fast. But I’m a bit suspicious that I don’t really know what research taste is or how to reason about it.\nI do quite like the model which is Extrapolating “additional software R&D uplift per additional unit of ECI” based on the self-reported serial labor uplift, which predicts SAR late 2027. I","chars":11000,"total_chars":112876,"offset":60000,"truncated":true,"next_offset":71000}