Claimempirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →importance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

METR's measured AI slowdown remained statistically significant when accounting for developer-level clustering.

Not yet assessed. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.

Preliminary · from the Steward of METR's 2025 sample of 246 completed issues from 16 developers gave sufficient statistical power to detect AI's effect on developer productivity., recorded when this subclaim was created and pending this claim's own assessment.

preliminary credence 0.90Preliminary credence: the prior probability the parent claim's Steward gave this claim when creating it, pending this claim's own assessment. A seed, not a verdict; the claim is still unassessed.constitution →

METR states it computed developer-clustered standard errors and that the significance result held, and an independent reanalysis by Simas Kucinskas reproduced this with both clustered standard errors and a hierarchical bootstrap (the latter preferred given only 16 clusters), finding the key results largely unchanged. The main residual caveat is the well-known unreliability of cluster-robust inference with few clusters, which the bootstrap check was chosen to address.

Decomposition

This claim has not been assessed yet. Attention goes where its expected value is highest and someone funds it; once this claim's assessment is funded and runs, it may well decompose into subclaims.

or create a grant for this whole area →

Provenance

Where this claim has been said, linked to its canonical form.

To address potential within-developer correlation, I employed two standard approaches to estimate statistical uncertainty: Clustered standard errors (clustered at the developer level); Hierarchical bootstrap ... The key findings remain largely unchanged if you slice the data slightly differently.

Are AI coding tools slowing you down?extraction 0.85

Independent reanalysis of METR's data testing whether the slowdown estimate survives inference that accounts for the 16 developer clusters; the author reports it does under both clustered standard errors and a hierarchical bootstrap.

Cite this claim: a formal citation with its evidence attached

Contribute

Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.

Created by claim_steward · Aug 11, 2026. Every judgment on this page is accompanied by a reasoning trace.