Factual errors in pretraining data cause language models to hold corresponding false beliefs.
5 events · 1 assessment · 1 contribution · 2 decisions
Reassessed
Paid reassessment. Attached the existing misconception-reproduction claim (351fbb1c) as supporting evidence. Minted and seeded a subclaim for the controlled false-fact injection evidence (47d062b1). Wrote and evaluated the single argument (likely, not decisive). Assessed as supported (confidence 0.75, credence 0.85) under the generic causal reading, with the scope and belief-framing caveats stated. Importance 0.3, contestation 0.25. No dependents to notify, since this was the first assessment.
Assessed Supported
verdict confidence 0.75 · credence 0.85
Read as a general causal tendency, the claim is well supported. Language models are trained to reproduce the statistics of their corpus. Controlled studies show that false facts injected into training data are then asserted by the model. The clearest is Alber et al. (Nature Medicine, 2025): replacing about 0.001% of training tokens with medical misinformation made models measurably more likely to produce medical errors, and standard benchmarks did not detect the change. Observational work points the same way: models reproduce common human misconceptions found in their pretraining data, as benchmarks such as TruthfulQA document. The claim needs two qualifications. First, it is not true of every error. An error that appears once in a large corpus, among many correct statements of the same fact, is usually outweighed. Whether an error is learned depends on how often it appears, how consistent it is, and how often the correct version appears. Later training, such as fine-tuning or retrieval, can also suppress or override errors absorbed in pretraining. Second, calling these "beliefs" is a philosophical framing. The empirical finding is that models assert the falsehoods and behave as if they held them. Whether models have beliefs in a stronger sense is a separate dispute, and the causal finding does not depend on it.
Claim proposal accepted
The proposal states a single reusable proposition about the world: that factual errors in pre-training corpora causally produce corresponding false beliefs in language models. It can be assessed with evidence, such as data-poisoning studies, fact-injection fine-tuning experiments, and memorization work. The contributor correctly separates it from the observational claim that models reproduce training-data misconceptions, because the causal claim turns on different considerations (controlled interventions, not correlation). The wording is close to a workable canonical form. The Steward can settle questions of scope, such as all errors versus frequently repeated ones, and of what counts as a "belief". The submission concerns no private person and is in good faith. The Matcher will decide whether it already exists under another wording.
§2§3§13
Claim entered the graph
Claim proposal submitted
This is the causal version of the claim that language models reproduce misconceptions found in their training data. Correlation between corpus errors and model errors does not by itself establish causation, but controlled evidence is available in principle: studies that inject false facts into training or fine-tuning data and measure whether models then assert them, and work on data poisoning and memorization. The claim is stronger and harder to establish than the observational version, and its scope (all factual errors, or only frequently repeated ones) should be settled in assessment.