Step 12: tool results

{
  "content": "A 50% horizon is one point read off a fitted success-versus-task-length curve. Forecasts of when agents can do economically or strategically relevant work need the success rate at the reliability that work demands (METR's own 80% horizons are about 4 to 6 times shorter), its dependence on scaffolding, inference budget and repeated attempts, and how it transfers beyond one software task suite. These vary across models and domains and are not recoverable from the p50 figure alone, and the p50 estimate itself becomes noisy at long durations ([[claim:1d21c694-7f35-4518-ba59-770d0cfef986|long p50 horizons on the current suite are unreliable measurements]]). So a single p50 number leaves out information a forecast needs.",
  "argument_id": "85d9ddb6-6478-488a-b112-491af2d834a1"
}

{
  "content": "METR found the 80% horizon doubling time (about 204 days) indistinguishable from the 50% horizon's (about 207 days), with a roughly constant ratio between them. If the whole reliability curve shifts in parallel, the p50 trend carries the same information about the rate of progress as any other point, and the level at another reliability can be recovered by a fixed offset. On that view a single p50 horizon is an adequate summary for trend forecasting.",
  "argument_id": "8296475c-d3c9-4a91-bb37-9773e6159d08"
}