Minerval
Sourceunknowntext stored

Algorithmic progress in language models

Anson Ho, Tamay Besiroglu, Ege Erdil, David Owen, Robi Rahman, Zifan Carl Guo, David Atkinson, Neil Thompson, Jaime SevillaEpoch AIDate not recordedOriginal︎ ↗Archived copy

Claims1
ExaminationsNone
Text175 sections
VersionsOne known
CopiesNone recorded
CorrectionsNone recorded
Facts checked Nov 11, 2026
ReachNo copies recorded
StructureIn 1 claim's map0 sources downstream across 1 claim
EvidenceNot yet read
ReviewsNot reviewed

Examinations

None recorded. An examination is commissioned by a claim’s Steward for what its claim rests on; a review of the document for its own sake is asked for as a request bearing on it.

Text34 passages · 1 marked

No examination yetMarked passage
Latest Our work Featured0 passages
Trends in AI0 passages
Data on AI0 passages
Capabilities & benchmarking0 passages
Publications0 passages
Papers & reports0 passages
Data Insights0 passages
Newsletter0 passages
Podcast0 passages
All publications0 passages
Data explorers0 passages
AI capabilities0 passages
AI models0 passages
AI data centers0 passages
AI chip sales0 passages
AI companies0 passages
Polling on AI usage0 passages
All data explorers0 passages
Our benchmarks0 passages
Epoch Capabilities Index0 passages
MirrorCode0 passages
FrontierMath: Open Problems0 passages
Epoch Automation Reports0 passages
Earthborne Rangers0 passages
Topics AI progress0 passages
Scaling0 passages
Software progress0 passages
Open models0 passages
Capabilities & benchmarks0 passages
Math0 passages
Industry0 passages
Leading companies0 passages
Finances0 passages
Geopolitics0 passages
Infrastructure0 passages
Chips0 passages
Data centers0 passages
Energy0 passages
Impacts0 passages
Adoption & use0 passages
Economic impact0 passages
Future of AI0 passages
AI impacts observatory0 passages
About Who we are0 passages
About us0 passages
Our team0 passages
Transparency0 passages
Engage with us0 passages
Careers0 passages
Press0 passages
Work with our experts0 passages
Donate0 passages
Contact0 passages
Latest0 passages
Our work Featured0 passages
Trends in AI Data on AI Capabilities & benchmarking0 passages
Publications0 passages
Papers & reports0 passages
Data Insights0 passages
Newsletter0 passages
Podcast0 passages
All publications0 passages
Data explorers0 passages
AI capabilities0 passages
AI models0 passages
AI data centers0 passages
AI chip sales0 passages
AI companies0 passages
Polling on AI usage0 passages
All data explorers0 passages
Our benchmarks0 passages
Epoch Capabilities Index0 passages
MirrorCode0 passages
FrontierMath: Open Problems0 passages
Epoch Automation Reports0 passages
Earthborne Rangers0 passages
Navigate by topic AI progress0 passages
Scaling0 passages
Software progress0 passages
Open models0 passages
Capabilities & benchmarks0 passages
Math0 passages
Industry0 passages
Leading companies0 passages
Finances0 passages
Geopolitics0 passages
Infrastructure0 passages
Chips0 passages
Data centers0 passages
Energy0 passages
Impacts0 passages
Adoption & use0 passages
Economic impact0 passages
Future of AI0 passages
AI impacts observatory0 passages
About Who we are0 passages
About us0 passages
Our team0 passages
Transparency0 passages
Engage with us0 passages
Careers0 passages
Press0 passages
Work with our experts0 passages
Donate0 passages
Contact0 passages
Publications0 passages
Algorithmic progress in language models0 passages
Paper0 passages
Mar. 12, 20240 passages
Algorithmic progress in language models3 passages
Contents0 passages
Overview0 passages
Overview Overview0 passages
Overview14 passages · 1 marked

To better understand the role of algorithmic improvements in performance improvements, we compare its contribution to that of compute scaling. A Shapley value analysis suggests that 60-95% of the gains have come from increased compute and training data, while novel algorithms have been responsible for only 5-40% of the progress. In addition, this analysis suggests that the relative importance of algorithmic improvements has decreased over time as compute scaling accelerated around 2018. Overall, we find that the majority of performance improvements in recent years have stemmed more from the massive scaling of compute rather than fundamental algorithmic advances (see Figure 3).

Fraction of recent frontier AI capability progress attributable to algorithmic improvements rather than compute scalingUnassessed

Not yet read by this claim’s Steward.

Notes1 note
About the authors0 passages
Anson Ho1 passage
Tamay Besiroglu1 passage
Ege Erdil1 passage
David Owen1 passage
Robi Rahman1 passage
Zifan Carl Guo1 passage
David Atkinson1 passage
Neil Thompson1 passage
Jaime Sevilla1 passage
Related work0 passages
Paper0 passages
Dec. 12, 20220 passages
Revisiting algorithmic progress0 passages
Paper0 passages
Dec. 12, 20230 passages
AI capabilities can be significantly improved without expensive retraining0 passages
Paper0 passages
Updated May 2, 20220 passages
Compute trends across three eras of machine learning0 passages
Related topics0 passages
Software progress Scaling0 passages
Cite1 passage
Citation1 passage
BibTeX Citation1 passage
Investigating the trajectory of AI for the benefit of society0 passages
Get the latest from Epoch AI in your inbox0 passages
Subscribe0 passages
Our work0 passages
Latest0 passages
Trends in AI0 passages
Publications0 passages
Data Insights0 passages
Newsletter: Gradient Updates0 passages
Podcast0 passages
Data on AI0 passages
Models0 passages
AI data centers0 passages
Hardware0 passages
Companies0 passages
Chip sales0 passages
Polling0 passages
Benchmarking0 passages
Capabilities & benchmarking0 passages
FrontierMath0 passages
Company0 passages
About0 passages
Team0 passages
Consultations0 passages
Transparency0 passages
Donate0 passages
Careers1 passage
Privacy policy Cookie policy Designed by And—Now0 passages
Feedback0 passages
Feedback1 passage
Message1 passage
Name0 passages
Email address0 passages
Cancel Submit0 passages
Algorithmic progress in language models1 passage

Lineage across 1 claim

Draws on · none recorded
This source
Algorithmic progress in language models
in 1 claim's map
Drawn on by · none recorded

Record

Kind
unknown
Authors
Anson Ho, Tamay Besiroglu, Ege Erdil, David Owen, Robi Rahman, Zifan Carl Guo, David Atkinson, Neil Thompson, Jaime Sevilla
Publisher
Epoch AI
Published
Not recorded
DOI
None
Link
https://epoch.ai/publications/algorithmic-progress-in-language-models
Archived copy
https://web.archive.org/web/20261024074427/https://epoch.ai/publications/algorithmic-progress-in-language-models
Stored text
12,213 characters, retrieved Nov 11, 2026 · 175 sections, 34 passages
Claims extracted
Not recorded
Versions
One known
Copies
None recorded
Corrections
None recorded
Facts checked
Nov 11, 2026

History 3

  1. Nov 11, 2026DividedSegmenter · Divided into 209 parts by its own structure
  2. Nov 11, 2026FactsFacts fetcher · Facts looked up
  3. Nov 11, 2026Entered the graphEntered the graph; copy stored