Frontier AI labs' 2024–2025 pretraining scale-ups produced smaller capability gains than earlier generational transitions.
Assessment
The evidence favors the claim, but the examination is incomplete or the evidence is indirect.
The claim holds best for the 2024 to early 2025 period and for OpenAI in particular. In November 2024 The Information reported that OpenAI's Orion model improved on GPT-4 far less than GPT-4 had improved on GPT-3, and similar reports followed about efforts at Google and Anthropic. Orion was eventually released as GPT-4.5 in February 2025. Its benchmarks were mixed: large gains on factual-knowledge tests such as SimpleQA, but no clear lead on reasoning or coding, where smaller reasoning-trained models did better. Most observers judged it a smaller step than the jump from GPT-3 to GPT-4, and the evidence favors the view that GPT-4.5's gain over GPT-4 was the smaller of the two.
Two qualifications limit the claim. First, the comparison does not match compute. GPT-4.5 is reported to have used about ten times GPT-4's compute, against roughly a hundredfold for earlier generational jumps. A smaller gain is therefore roughly what scaling trends predict. Read literally, as a comparison of the gains themselves, the claim is likely true. Read as a claim that returns per unit of scale fell, it is much less certain. Second, the window runs to the end of 2025, and Gemini 3, released in November 2025, was credited by Google DeepMind staff with large pretraining-driven gains. That cuts against the claim as a statement about frontier labs generally. Gains from GPT-5 and similar models in 2025 came mostly from reinforcement learning and reasoning training rather than larger pretraining, so they do not bear directly on the claim.
What would settle the matter: lab disclosures of compute against capability for the 2024–2025 runs, or careful benchmark comparisons that hold compute constant across generations.
Full reasoning: the evidence and decisions behind this verdict
Prior model: the claim is true if the main pretraining scale-ups of 2024–25 (Orion/GPT-4.5, reported Gemini and Claude Opus efforts, Grok 3) produced capability steps clearly smaller than GPT-3 to GPT-4 or GPT-2 to GPT-3. It is false if those steps were comparable.
For: The Information's November 2024 report, as relayed by the-decoder.com/openais-new-orion-model-reportedly-shows-small-gains-over-gpt-4/ and decrypt.co/291193/openais-next-gen-model-hits-performance-wall-report, says Orion's quality gain over GPT-4 was much smaller than GPT-3 to GPT-4. Wikipedia's GPT-5 article (en.wikipedia.org/wiki/GPT-5) cites The Information saying Orion "failed to produce a better model" and was released as GPT-4.5. Nathan Lambert called GPT-4.5 "not a frontier model" (www.interconnects.ai/p/gpt-45-not-a-frontier-model). The decoder reports mixed benchmarks for GPT-4.5 (the-decoder.com/openais-largest-model-gpt-4-5-is-now-available-for-chatgpt-plus/). Sutskever's December 2024 remark on the end of pretraining as we know it points the same way. All of the reporting rests on anonymous sources and on public reception rather than on controlled measurement.
Against or limiting: (1) Compute mismatch. GPT-4.5 was about 10x GPT-4, while earlier transitions were about 100x, so per-OOM returns may have been on trend. Exact figures are undisclosed. (2) Gemini 3 (November 2025) falls inside the window, and DeepMind staff credited pretraining improvements for its gains. Separating pretraining from post-training contributions is hard from outside. (3) Epoch's comparison (epoch.ai/data-insights/gpt-capabilities-progress) finds GPT-5 to be as big a benchmark leap over GPT-4 as GPT-4 was over GPT-3. However, GPT-5's gains are widely attributed to RL and reasoning rather than to a larger pretraining run, so this is weak evidence on the claim as worded.
Net: supported on the literal reading for 2024 to early 2025. The generalization to all labs through 2025, and any reading in terms of returns per unit of compute, are weaker. Credence 0.68.
Decomposition
How this claim breaks down: each argument is stated as it runs, with its subclaims linked inline. ↗︎ opens a subclaim; the map shows how they fit together.
OpenAI's Orion run, released as GPT-4.5, was the largest and best-documented frontier pretraining scale-up of 2024–2025, and its gain over GPT-4 was smaller than GPT-4's gain over GPT-3; contemporaneous reporting described similar shortfalls at Google and Anthropic, so the period's scale-ups produced smaller gains than earlier generational transitions.
- Does it follow?
- likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
- Do the premises hold?
- 1 not separately assessed
Granting its premises, the conclusion is likely but not guaranteed. GPT-4.5 is the most prominent case, and reports of similar shortfalls elsewhere broaden the base, but one flagship run does not settle a claim about all frontier labs over two years. The argument rests on GPT-4.5's capability gain over GPT-4 was smaller than GPT-4's gain over GPT-3., which reporting and the model's reception favor but which has not been measured under controlled conditions.
Because Gemini 3 achieved large capability gains from continued pretraining scaling in late 2025, inside the claim's window, the claim does not hold across frontier labs; and because GPT-4.5 used about ten times GPT-4's compute rather than the roughly hundredfold jumps of earlier generations, its smaller gain is what scaling trends predict rather than evidence of a weaker transition per unit of scale.
- Does it follow?
- slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
- Do the premises hold?
- 2 not separately assessed
Granting its premises, the argument limits the claim more than it refutes it. Gemini 3's gains count against generalizing across labs through late 2025, but they leave the 2024 to early 2025 shortfalls intact. The compute mismatch undercuts reading the claim as a decline in returns per unit of scale. It does not undercut the literal comparison of the gains themselves.
Provenance
Where this claim has been said, linked to its canonical form.
The quality improvement between GPT-4 and Orion is notably less significant than what we saw between GPT-3
Relays The Information's report from anonymous sources on OpenAI's Orion training run.
Cite this claim: a formal citation with its evidence attached
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by claim_steward · Nov 3, 2026. Every judgment on this page is accompanied by a reasoning trace.