AI coding assistants substantially speed up developers on many programming tasks.
Assessment
Evidence favors the claim, but the chain is incomplete or the sources are secondary.
Controlled experiments have repeatedly found large speedups from AI coding assistants in a wide range of settings. GitHub's 2022 randomized experiment found Copilot users finished a self-contained coding task about 55% faster, and large randomized field deployments at Microsoft, Accenture, and a third firm found roughly 26% more tasks completed in ordinary enterprise work, with a six-week trial at ANZ Bank reporting completion roughly 42% faster. Gains in these studies were largest for less experienced developers and for work in unfamiliar territory, conditions common in real software work.
The main counterevidence is a 2025 randomized trial by METR in which experienced open-source maintainers working on large codebases they knew well completed tasks about 20% slower with AI tools, even while believing the tools had sped them up. That result shows the speedup is not universal: it can vanish or reverse for expert work in mature, familiar codebases. It bounds the claim rather than refuting it, since the favorable evidence covers greenfield tasks, enterprise development, and less experienced developers, which plausibly constitute many programming tasks. There is also indication that tools in early 2026 sped developers up more than early-2025 measurements suggested, though the follow-up data behind that indication suffered from selection problems.
The credible disagreement is therefore about scope, not existence: how representative the settings with large measured gains are of real-world programming as a whole, and whether measured speedups on individual tasks translate into overall productivity. Continued randomized measurement across task types and experience levels, of the kind METR has attempted to run, would sharpen the answer.
Full reasoning: the evidence and decisions behind this verdict
The verdict rests on weighing two rigorous but divergent bodies of controlled evidence.
For the claim: Peng et al.'s randomized experiment (arXiv:2302.06590) had 95 freelance developers build a JavaScript HTTP server; the Copilot group finished 55.8% faster (95% CI 21–89%), about 71 versus 161 minutes. Cui et al.'s randomized field experiments with roughly 5,000 developers at Microsoft, Accenture, and an anonymous firm found about 26% more completed tasks (measured mainly as pull requests) for the Copilot group, with the largest gains among less experienced and recently hired developers (see MIT Sloan's summary, mitsloan.mit.edu/ideas-made-to-matter/how-generative-ai-affects-highly-skilled-workers, and the preview at mit-genai.pubpub.org/pub/v5iixksv). A six-week ANZ Bank trial (Chatterjee et al.) reported tasks completed 42.4% faster, with gains at every skill level. These span lab, enterprise field, and bank settings, which is what "many programming tasks" requires.
Against: METR's RCT (arXiv:2507.09089) found 16 experienced open-source maintainers, working on their own large, familiar repositories with early-2025 tools, took 19% longer with AI allowed, while estimating afterward that AI had sped them up about 20%. This is methodologically strong and is the reason the verdict is not simply verified: it demonstrates a real, economically important class of work where the claim fails.
Reconciliation: the heterogeneity finding, that gains concentrate among less experienced developers and unfamiliar codebases, makes the two bodies of evidence consistent rather than contradictory. The claim as worded asserts speedups on many tasks, not all tasks or on net across all work; the METR result bounds it without negating it. METR's February 2026 update (metr.org/blog/2026-02-24-uplift-update/) reports that its follow-up experiment could not produce reliable estimates because 30–50% of developers declined to submit tasks they would have to do without AI, but states the team believes developers are likely more sped up by early-2026 tools than the early-2025 estimate indicated; this weakly raises credence but is not load-bearing.
Caveats weighed: the 55.8% result comes from one self-contained greenfield task; the 26% figure uses pull-request counts as an output proxy and reached statistical significance clearly only at Microsoft; speed is not quality, and none of the favorable studies measure downstream defect or review costs. The alternative reading is contested, and it was seriously considered: credible randomized evidence points in opposite directions. It was set aside because the disagreement, examined closely, concerns the stronger claim that AI speeds up developers on most or all work, including expert work in mature codebases; on the claim as actually worded, the favorable evidence covers a wide enough range of settings that the balance favors it.
What would change the conclusion: replication of METR-style slowdowns across a broader range of developers and task types (which would push toward contested or contradicted), or reliable 2026-era randomized results showing consistent speedups even for experienced maintainers (which would push toward verified). Credence 0.75 reflects the residual scope question, not doubt about the individual experimental results.
Decomposition
How this claim breaks down: each argument is stated as it runs, with its subclaims linked inline. ↗︎ opens a subclaim; the map shows how they fit together.
Because a controlled experiment found Copilot users finished a coding task about 55% faster and large randomized field deployments found about 26% more tasks completed, substantial speedups appear in both lab and real enterprise settings. Given that gains are concentrated among less experienced developers and unfamiliar codebases, which are common working conditions, and that tools sped developers up more by early 2026 than early-2025 estimates indicated, the speedup plausibly extends to many programming tasks.
The inference goes through if the experimental results generalize: speedups documented in lab, enterprise, and banking settings do plausibly cover many programming tasks. The weight rests on the large field experiments finding about 26% more tasks completed, since that is the only evidence from ordinary real-world work at scale, and it relies on pull-request counts as an output proxy; the 55% lab result is well-established but comes from a single greenfield task. The caveat is scope: the argument establishes speedups in these settings, and reaches "many programming tasks" only insofar as the settings where gains concentrate are representative of real work.
Because a randomized trial found experienced open-source developers completed tasks about 20% slower with AI tools in early 2025, substantial speedups do not hold in at least one economically important class of real-world work, complex changes to large codebases the developer knows well, which weighs against the claim that speedups cover many programming tasks.
The premise is well-supported and the inference is sound as far as it reaches: the randomized finding that experienced open-source developers were slowed about 20% establishes a real class of work where the claimed speedup fails. The caveat is that this bounds rather than refutes the claim, which asserts speedups on many tasks, not all: expert work in large, familiar codebases is one setting, measured with early-2025 tools, and the study's own authors present it as a snapshot rather than a general result.
Provenance
Where this claim has been said, linked to its canonical form.
While the extent differs, the general assertion of Copilot increasing developers productivity is also supported by other, more independent sources.
A literature review of Copilot's productivity impact, endorsing in its own voice the conclusion that Copilot increases developer productivity across multiple independent studies.
Cite this claim: a formal citation with its evidence attached
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by claim_steward · Aug 11, 2026. Every judgment on this page is accompanied by a reasoning trace.