Sourceunknowntext stored

TASTE: Can AI Models Judge AI Safety Research Proposals?

Author not recordedDate not recordedOriginal︎ ↗Archived copy

Claims1

ExaminationsNone

Text7 sections

VersionsOne known

CopiesNone recorded

CorrectionsNone recorded

Facts checked Nov 11, 2026

ReachNo copies recorded

StructureIn 1 claim's map0 sources downstream across 1 claim

EvidenceNot yet read

ReviewsNot reviewed

Examinations

None recorded. An examination is commissioned by a claim’s Steward for what its claim rests on; a review of the document for its own sake is asked for as a request bearing on it.

Text25 passages · 1 marked

No examination yetMarked passage

Alignment Science Blog1 passage

Hasan Baig 1 , Hailey Joren 2 , Joe Benton 20 passages

August 28, 20260 passages

1 Anthropic Fellows Program; 2 Anthropic3 passages

Background16 passages

Evaluating Models’ Research Judgment2 passages · 1 marked

Model performance on TASTE by release date and model provider. Fable 5 achieves 60%, and almost all models perform within 2 standard deviations of chance. Opus 5 and GPT-5.6-Sol perform near chance on TASTE despite being at the frontier on general agentic benchmarks. Per-model confidence intervals span roughly ±10 percentage points, as we have a limited number of preference pairs making it difficult to draw conclusions about relative model performance.

Current AI systems have research taste comparable to top human AI researchers.Doubtfuldenies

Not yet read by this claim’s Steward.