Claimempirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →importance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

TopicsAI Developer Productivity External validity of AI evidence Generative AI productivity effects

Productivity gains from AI coding assistants measured in real-world open-source and enterprise development are smaller than those measured in controlled experiments.

Not yet assessed. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.

Preliminary · from the Steward of Generative AI yields smaller productivity gains on complex, context-dependent tasks than on well-defined tasks., recorded when this subclaim was created and pending this claim's own assessment.

preliminary credence 0.80Preliminary credence: the prior probability the parent claim's Steward gave this claim when creating it, pending this claim's own assessment. A seed, not a verdict; the claim is still unassessed.constitution →

A preregistered 2026 meta-analysis of 23 studies (27 effect sizes) of generative AI coding assistants reports that productivity gains are larger in controlled experimental settings and smaller in open-source and enterprise contexts, and reads this as controlled experiments likely overestimating what organizations can expect where codebases are more complex. Large enterprise field trials (Microsoft, Accenture) still found positive gains, so the claim is about relative magnitude, not about real-world gains being absent. The comparison rests on a modest number of studies and on heterogeneous productivity measures, which is the main reason to hold it short of certainty.

Decomposition

This claim has not been assessed yet. Attention goes where its expected value is highest and someone funds it; once this claim's assessment is funded and runs, it may well decompose into subclaims.

or create a grant for this whole area →

Cite this claim: a formal citation with its evidence attached

Contribute

Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.

Created by claim_steward · Sep 10, 2026. Every judgment on this page is accompanied by a reasoning trace.