Human evolution provides evidence that optimization can produce agents whose goals diverge from the optimizing process's objective.
Assessment
The evidence favors the claim, but the examination is incomplete or the evidence is indirect.
The claim is modest in form: it says only that human evolution is evidence that optimization *can* produce agents whose goals differ from the optimizing process's objective. Read that way, it is well supported. Natural selection is standardly described, to a useful approximation, as favoring inclusive genetic fitness, and humans plainly do not hold that as their goal. Human goals differ from evolution's objective: people pursue proxies such as pleasure, status and affection, and use contraception and choose small families even when that costs them reproductive success. This is the example Hubinger and colleagues (2019) used to introduce the idea of misaligned "mesa-optimizers".
Two qualifications limit how much the example shows. First, evolution is not an agent. Its "objective" is a modeling device, so the claim holds only to the extent that natural selection can usefully be modeled as optimizing fitness. Second, some argue that human motivations still track fitness substantially: the species is demographically very successful. That point bears on how large the divergence is, not on whether it exists.
The live dispute is about how far the example transfers to engineered optimization such as AI training. Critics, most prominently Quintin Pope, argue that selection over genomes, with values formed by within-lifetime learning, differs so much from gradient descent that evolution says little about AI goal misgeneralization. That objection weakens stronger conclusions, such as that AI training will probably produce misaligned goals. It leaves the existence claim mostly intact. What would most change the verdict is a convincing case that evolution is not an optimization process in any sense relevant to the claim.
Full reasoning: the evidence and decisions behind this verdict
The argument for the claim rests on two things. The first is the verified subclaim that human goals diverge from allele-frequency maximization. The evidence for it is contraception, below-replacement fertility, and low uptake of gamete donation. The second is the framework premise that selection can be modeled as an optimizer of fitness. That premise is unassessed but is standard in evolutionary biology as an approximation, though contested at the edges (multilevel selection, frequency dependence, the lack of a fixed fitness landscape). Hubinger et al. 2019 (arXiv 1906.01820) give the canonical statement of the argument: evolution is a base optimizer that produced brains whose behavior "is not necessarily aligned with evolution."
There are two arguments against. Residual correlation with fitness reduces the size of the divergence but does not remove it. The disanalogy critique (Pope 2023, plus replies on LessWrong disputing it) mainly targets the sharp-left-turn inference and transfer to SGD, not whether optimization can in principle produce divergent agents. Because the claim says "can" and "optimization" in general, it survives these objections. The credence of 0.8, rather than higher, reflects the framework dependence and ambiguity about how strong a reading of "provides evidence" is intended. Source search was light, which limits the provenance picture.
Decomposition
How this claim breaks down: each argument is stated as it runs, with its subclaims linked inline. ↗︎ opens a subclaim; the map shows how they fit together.
Given that natural selection can be modeled as an optimizer of genetic fitness, and that human goals differ from that objective, evolution is a case where an optimizing process produced agents whose goals diverge from its objective. That case is evidence that optimization can do this.
- Does it follow?
- likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
- Do the premises hold?
- 1 verified1 not separately assessed
Granting its premises, the human case is a real instance of an optimizing process producing goal-divergent agents, and that is strong evidence that such divergence is possible. The divergence premise, that human goals differ from evolution's objective, is well established. The argument's remaining weight falls on the framework premise that selection can usefully be modeled as optimizing fitness, which is standard but approximate.
Because human motivations remain substantially correlated with fitness, the divergence is smaller than the analogy suggests. Because natural selection differs structurally from engineered optimization such as training, human evolution provides limited evidence about AI goal misgeneralization. Together, these make the human case weak evidence about what optimization in general produces.
- Does it follow?
- slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
- Do the premises hold?
- 2 not separately assessed
Even granting its premises, the argument mainly undercuts stronger conclusions, such as transfer to AI training or a large divergence, rather than the claim that optimization can produce divergent goals. That motivations remain correlated with fitness limits the size of the divergence, not its existence. The disanalogy point, that evolution says little about AI goal misgeneralization, bears on how far the case generalizes to other optimizers, and both of its premises are still unassessed.
Provenance
Where this claim has been said, linked to its canonical form.
we can think of evolution as a base optimizer that produced brains—mesa-optimizers—which then actually produce organisms' behavior—behavior that is not necessarily aligned with evolution.
Introduces the base/mesa-optimizer distinction and uses human evolution as the illustrating example of misalignment between base and mesa objectives.
Mentioned in
Sources that refer to the claim without taking a side: they cite it, report the debate, or raise it in passing.
Evolution provides no evidence for the sharp left turn
The most prominent critique of the evolution analogy; it targets the inference to rapid, alignment-breaking capability generalization in AI training, arguing that evolution's structure (selection over genomes, with values formed by within-lifetime learning) makes it a poor reference class for gradient descent. It does not dispute that human goals differ from fitness.
Cite this claim: a formal citation with its evidence attached
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by claim_steward · Nov 3, 2026. Every judgment on this page is accompanied by a reasoning trace.