Sourceunknownno stored text
Many SWE-bench-Passing PRs Would Not Be Merged into Main
Author not recordedDate not recordedOriginal︎ ↗
Claims2
ExaminationsNone
TextNot stored
VersionsOne known
CopiesNone recorded
CorrectionsNot checked
Facts not yet checkedReachNo copies recorded
StructureIn 2 claims' maps0 sources downstream across 2 claims
EvidenceNot yet read
Examinations
None recorded.
Text
No stored copy. Only the passages claims quote are recorded.
As quoted
our results indicate that a naive interpretation of benchmark scores may lead one to overestimate how useful agents are without more elicitation or human feedback.
Time-horizon trends measured on benchmark software tasks generalize to real-world software work.
Not yet read by this claim’s Steward.
As quoted
our results indicate that a naive interpretation of benchmark scores may lead one to overestimate how useful agents are without more elicitation or human feedback.
AI benchmarks overestimate models' capabilities on real-world tasks.
Not yet read by this claim’s Steward.
Lineage across 2 claims
Draws on · none recorded
This source
Drawn on by · none recordedMany SWE-bench-Passing PRs Would Not Be Merged into Main
in 2 claims' maps
Record
- Kind
- unknown
- Authors
- Not recorded
- Publisher
- Not recorded
- Published
- Not recorded
- DOI
- None
- Link
- https://metr.org/notes/2026-03-10-many-swe-bench-passing-prs-would-not-be-merged-into-main/
- Archived copy
- None found
- Stored text
- None; quoted passages only
- Claims extracted
- Not recorded
- Versions
- One known
- Copies
- None recorded
- Corrections
- Not yet checked
- Facts checked
- Not yet
History 1
- Nov 2, 2026Entered the graphEntered the graph; no copy stored