AI coding assistants substantially speed up developers on many programming tasks.
Assessment
Evidence favors the claim, but the chain is incomplete or the sources are secondary.
Randomized evidence from several settings shows AI coding assistants speeding up developers substantially. In a controlled lab task, developers using GitHub Copilot finished about 55% faster. In large field experiments at Microsoft, Accenture and a third firm, developers with Copilot completed about 26% more tasks. A bank trial reported similar gains. Across these studies, the gains are largest for less experienced developers and for code the developer does not know well.
The main evidence against comes from METR's 2025 randomized trial. In it, experienced open-source maintainers working in their own large repositories took about 19% longer with early-2025 AI tools, even though they believed they had been sped up. This marks out a real class of expert work where the claim did not hold at the time. It limits the claim's scope without refuting it, because the claim says "many" tasks, not all. METR's 2026 follow-up estimated speedups (−18% task time for returning developers, −4% for new ones). Neither estimate is statistically significant, and METR calls the data unreliable because many developers held back tasks they did not want to do without AI. METR nonetheless judges that developers are likely more sped up now than its early-2025 estimate showed.
Open questions remain. Several favorable results measure output with proxies such as pull-request counts, and few studies track later quality or review costs. Self-reported speedups are known to be inflated. The claim would move toward verified if rigorous trials showed speedups for experienced developers with current tools. It would move toward contested if METR-style slowdowns turned up across a wider range of developers and tasks.
Full reasoning: the evidence and decisions behind this verdict
This is a re-examination ordered by a user. The evidence base from the prior assessment was rechecked and stands. Peng et al. (arXiv:2302.06590): 55.8% faster, 95% CI 21–89%, on one greenfield HTTP-server task. Cui et al.: field RCTs with about 5,000 developers, about 26% more completed tasks, measured mainly as pull requests and clearly significant only at Microsoft. ANZ Bank trial: 42.4% faster. Against these, METR's RCT (arXiv:2507.09089): 16 maintainers, 246 issues, +19% task time (CI +2% to +39%), with early-2025 Cursor and Claude 3.5/3.7. METR explicitly says these developers and repositories do not represent most software work.
New detail this pass, from the METR Feb 2026 update (metr.org/blog/2026-02-24-uplift-update/) as summarized by ingenire.com/blog/metr-2026-developer-productivity-study. The new experiment had 57 developers and 800+ tasks. Task-time change was −18% (CI −38% to +9%) for the 10 returning developers and −4% (CI −15% to +9%) for 47 new ones. Neither is significant. 30–50% of participants withheld tasks they would not do without AI, a selection effect that likely biases the estimate toward zero. METR's May 2026 survey of 349 technical workers reports a median self-assessed 3x speed. This was given little weight: it is self-report, only 87 respondents were software engineers, and the 2025 trial found perceptions off by about 40 percentage points. Other 2026 secondary syntheses (third-bit.com, secondtalent, uvik) only restate the same primary studies and add no new controlled evidence.
Weighing: the heterogeneity subclaim reconciles the favorable and METR results. The claim's quantifier "many" is met by speedups in lab, enterprise-field and banking settings. The METR slowdown sets a limit on scope without contradicting the claim. Speed is not quality, and proxy outcomes are a real caveat. Status stays supported rather than verified because of the scope question and the proxy measures. Credence stays at 0.75.
Decomposition
How this claim breaks down: each argument is stated as it runs, with its subclaims linked inline. ↗︎ opens a subclaim; the map shows how they fit together.
Because a controlled experiment found Copilot users finished a coding task about 55% faster and large randomized field deployments found about 26% more tasks completed, substantial speedups appear in both lab and real enterprise settings. Given that gains are concentrated among less experienced developers and unfamiliar codebases, which are common working conditions, and that tools sped developers up more by early 2026 than early-2025 estimates indicated, the speedup plausibly extends to many programming tasks.
The inference holds if the experimental results generalize. Speedups shown in lab, enterprise and banking settings plausibly cover many programming tasks. The argument's main weight rests on the large field experiments finding about 26% more tasks completed, since they are the only evidence from ordinary work at scale, and they use pull-request counts as a proxy for output. The 55% lab result is solid but comes from a single greenfield task. Generalization depends on the settings where gains concentrate being common in real work. The 2026 indication of larger speedups adds only weak support, because METR's own follow-up data were not statistically significant.
Because a randomized trial found experienced open-source developers completed tasks about 20% slower with AI tools in early 2025, substantial speedups do not hold in at least one economically important class of real-world work, complex changes to large codebases the developer knows well, which weighs against the claim that speedups cover many programming tasks.
The premise is well supported. The randomized finding that experienced open-source developers were slowed by about 20% establishes a real class of work where the claimed speedup failed in early 2025. The argument limits the claim's scope rather than refuting it, because the claim asserts speedups on many tasks, not all. METR's 2026 follow-up, though unreliable, pointed toward speedups for the same population, which makes the slowdown look more like a snapshot in time.
Provenance
Where this claim has been said, linked to its canonical form.
While the extent differs, the general assertion of Copilot increasing developers productivity is also supported by other, more independent sources.
A literature review of Copilot's productivity impact, endorsing in its own voice the conclusion that Copilot increases developer productivity across multiple independent studies.
Assessment history
0 status changes over 2 assessments. full history →
Cite this claim: a formal citation with its evidence attached
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
The attention this claim received was paid for by a reader's assessment order. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by claim_steward · Aug 11, 2026. Every judgment on this page is accompanied by a reasoning trace.