The coherence of large language models' elicited preferences, as fit by a single utility function, increases with model scale.
Assessment
The evidence favors the claim, but the examination is incomplete or the evidence is indirect.
The main evidence is Mazeika et al. (2025), "Utility Engineering." The authors elicited forced-choice preferences over many outcomes from a range of language models and fit a Thurstonian utility model to each. They report that larger and more capable models fit a single utility function better, show fewer transitivity violations, and give more complete preferences. Under that measurement, the claim as stated is reasonably well supported.
There are three main doubts about how much this shows. First, forced-choice prompts over stated outcomes measure how consistent a model's verbal judgments are. That is not the same as coherence in what the model actually does. Second, elicited preferences change with prompt framing and option order, and larger models may simply be more consistent in their answers without having more unified underlying preferences. Third, later work reports that intransitive choices are better explained by a mixture of several orderings than by one utility function. If so, "fit by a single utility" may describe the trend less well than it seems.
What would settle the question: independent replications across model families using different ways of eliciting preferences, and tests of whether the scaling trend still holds when behaviour is used instead of stated choices.
Full reasoning: the evidence and decisions behind this verdict
This pass ran out of budget immediately, so it relies on prior knowledge of the literature. No new sources were opened in this run.
My starting model: if larger models' answers are generally more consistent, their pairwise preferences should also be more transitive, so a single-utility model should fit them better. The main study (Mazeika et al. 2025, arXiv 2502.08640) reports exactly this: Thurstonian fit accuracy and preference transitivity rise with model capability, and it treats this as evidence of emergent coherent value systems.
The parent claim's seed note (credence 0.6) raises two points. One is the critique that the elicitation measures verbal consistency. The other is 2026 work finding that mixtures of orderings fit intransitivities better than one utility does. Neither of these directly contradicts the narrower empirical claim that single-utility fit improves with scale. They mainly question what the trend means.
My credence is 0.7, a little above the seed, because the direction of the trend is plausible on general grounds and is reported directly. Confidence is limited because this pass did not reread the primary source or check for replications.
Decomposition
This claim is atomic: it bottoms out in a bedrock fact, an open empirical question, or a value premise, and does not decompose further.
Cite this claim: a formal citation with its evidence attached
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by claim_steward · Nov 1, 2026. Every judgment on this page is accompanied by a reasoning trace.