Sourcepreprinttext stored

Measuring AI Ability to Complete Long Software Tasks

Kwa, Thomas, West, Ben, Becker, Joel, Deng, Amy, Garcia, Katharyn, Hasin, Max, Jawhar, Sami, Kinniment, Megan, Rush, Nate, Von Arx, Sydney, Bloom, Ryan, Broadley, Thomas, Du, Haoxing, Goodrich, Brian, Jurkovic, Nikola, Miles, Luke Harold, Nix, Seraphina, Lin, Tao, Painter, Chris, Parikh, Neev, Rein, David, Sato, Lucas Jun Koba, Wijk, Hjalmar, Ziegler, Daniel M., Barnes, Elizabeth, Chan, LawrencearXiv.orgPublished Mar 18, 2025Original︎ ↗Archived copy

Claims1

ExaminationsNone

Text1 section

VersionsOne known

CopiesNone recorded

CorrectionsNone recorded

Facts checked Nov 14, 2026

ReachNo copies recorded

StructureIn 1 claim's map0 sources downstream across 1 claim

EvidenceNot yet read

ReviewsNot reviewed

Examinations

None recorded. An examination is commissioned by a claim’s Steward for what its claim rests on; a review of the document for its own sake is asked for as a request bearing on it.

Text45 passages · 0 marked

No examination yetMarked passage

No claim quotes it yet.

Not found in the text 1

As quoted

However, models' 80% time horizons are 4-6x shorter, suggesting that even models that sometimes succeed on difficult and diverse tasks cannot reliably perform tasks of moderate length.

A single 50% task-completion horizon inadequately summarizes AI-agent capability for forecasting.Supportedmentions

Not yet read by this claim’s Steward.

Lineage across 1 claim

Draws on · none recorded

This source

Measuring AI Ability to Complete Long Software Tasks

in 1 claim's map

Drawn on by · none recorded

Record

Kind: preprint
Authors: Kwa, Thomas, West, Ben, Becker, Joel, Deng, Amy, Garcia, Katharyn, Hasin, Max, Jawhar, Sami, Kinniment, Megan, Rush, Nate, Von Arx, Sydney, Bloom, Ryan, Broadley, Thomas, Du, Haoxing, Goodrich, Brian, Jurkovic, Nikola, Miles, Luke Harold, Nix, Seraphina, Lin, Tao, Painter, Chris, Parikh, Neev, Rein, David, Sato, Lucas Jun Koba, Wijk, Hjalmar, Ziegler, Daniel M., Barnes, Elizabeth, Chan, Lawrence
Publisher: arXiv.org
Published: Mar 18, 2025
DOI: None
Link: https://arxiv.org/pdf/2503.14499
Archived copy: https://web.archive.org/web/20261106155829/https://arxiv.org/pdf/2503.14499
Stored text: 126,707 characters, retrieved Nov 14, 2026 · 1 sections, 45 passages
Claims extracted: Not recorded
Versions: One known
Copies: None recorded
Corrections: None recorded
Facts checked: Nov 14, 2026

History 3

Nov 14, 2026DividedSegmenter · Divided into 45 parts by its own structure
Nov 14, 2026FactsFacts fetcher · Facts looked up
Nov 14, 2026Entered the graphEntered the graph; copy stored