The best frontier AI models are approaching the quality of experienced industry professionals' work deliverables.
3 events · 1 assessment · 1 decision
Structured and assessed
First pass. Created three subclaims: the GDPval head-to-head results, GDPval's measurement validity (attached as an assumption), and RLI end-to-end completion (contradicts). Linked the existing claims on the linear win-rate trend and on one-shot task specification. Built two arguments. Assessed as supported (0.7, credence 0.75) because the claim's scope is ambiguous. Importance 0.65, contestation 0.55. No dependents to notify. The budget was limited, so the 2026 figures from aggregators were not verified against primary sources.
Assessed Supported
verdict confidence 0.70 · credence 0.75
The claim comes from OpenAI's GDPval benchmark (October 2025), in which experienced professionals in 44 occupations wrote realistic work tasks and other experts compared AI and human deliverables without knowing which was which. At publication the best model, Claude Opus 4.1, was rated as good as or better than the expert in about 48% of comparisons, which the authors described as approaching parity. Later results moved past that point: OpenAI reported 70.9% wins-or-ties for GPT-5.2 Thinking in December 2025, and leaderboards in 2026 list launch-post figures above 80%. On the task type GDPval measures, the claim is borne out and, if anything, understated, since by late 2026 the best models match or beat experts in most comparisons (the head-to-head GDPval results). The real disagreement is over scope. GDPval tasks are fully specified, self-contained, one-shot digital deliverables. They leave out gathering context, working iteratively with clients, tacit knowledge, and physical work (the task-specification limitation). Measured against real end-to-end projects, the picture is weaker. On the Remote Labor Index, which uses real paid freelance projects judged by whether a client would accept the result, the best agents completed 2.5% of projects in late 2025. Secondary reports put the figure at about 16% by mid-2026: still a small minority, but rising (the end-to-end completion finding). So the claim holds when read as a statement about the quality of well-specified professional deliverables. It does not hold when read as a statement that models can do professionals' work in general. Whether GDPval validly measures professional deliverable quality is the main open question. Independent replications of the GDPval figures and further end-to-end evaluations would settle more of it.
Claim entered the graph