Complex risk analyses historically exhibit error rates exceeding their claimed risk bounds
Assessment
Evidence favors the claim, but the chain is incomplete or the sources are secondary.
Read as a recurring historical pattern rather than a universal law, the claim is well supported: in several of the most heavily resourced risk analyses of their eras, the claimed probability bound was later exceeded by the realized failure rate, and the source of the gap lay in the analysis itself rather than in bad luck. The cleanest case is spaceflight, where pre-Challenger estimates put failure near one in 100,000 per flight against a realized rate on the order of one in 100. The pattern recurs in nuclear risk assessments that predicted core-damage frequencies far below the observed accident record and in bank value-at-risk models that understated extreme-loss frequency before the 2008 crisis. A structural floor reinforces the pattern: because complex technical work carries its own non-trivial error and retraction rate, any claimed bound set far below that floor is exceeded by the probability that the analysis is simply wrong.
The main qualification is scope. It is common for regulatory risk assessments to build in conservative assumptions that push estimates above the central risk, so the claim is false if read universally, as if all complex risk analyses understate risk. That practice, however, belongs to a different genre, chemical, environmental, and public-health screening, than the probabilistic engineering and financial models in which the exceedance cases arise, so it limits how far the pattern generalizes without rebutting it where very small bounds were claimed. A selection effect also remains unavoidable: exceeded bounds produce accidents and inquiries while bounds that held produce nothing, and no systematic calibration sample of risk analyses exists. The verdict is therefore supported rather than verified: the evidence is a set of salient, well-documented, unrebutted cases, not a systematic sample. A calibration study over a defined population of risk analyses would resolve it in either direction.
Full reasoning — evidence and decisions behind this verdict
Trigger: the contradicting subclaim regulatory risk assessments commonly use conservative assumptions that overstate risk received its first assessment (supported, confidence 0.72, credence 0.75) on the modest reading that health-protective conservative defaults are common and push estimates above central estimates, while the magnitude and systematicity of any net overstatement remains contested. This is materially confirmatory for the parent, not disruptive: the prior assessment already treated the conservatism point as credible and scope-limiting rather than pattern-contradicting, and the accompanying domain distinction sharpens that reading.
The supporting cases are unchanged and unrebutted. (1) Space Shuttle: pre-Challenger management estimates of ~1 in 100,000 per flight (Rogers Commission record, Feynman's appendix) against a realized loss rate of 2 in 135 and NASA's own retrospective placing early-flight risk near 1 in 9. The bound was wrong by roughly three orders of magnitude, and the error was in the analysis process. (2) Nuclear PRA: predicted core-damage frequencies (order 1e-6 per reactor-year at the plant level; ~2e-5 fleet-level) against actuarial severe-accident frequencies from the historical record on the order of 1e-4 to 1e-3. Real but softer, because the comparison depends on reference-class choices. (3) Value-at-risk: the clustering of VaR exceedances at major banks in 2007-2008 shows the same signature of claimed tail bounds exceeded in practice. (4) Structurally, error and retraction rates in peer-reviewed technical work (~1e-4 to 1e-2) put a floor under the probability that any complex analysis is itself flawed; this is the point the Ord, Hillerbrand, and Sandberg paper (Journal of Risk Research, 2010) builds on.
How the assessed contradicting subclaim weighs. Its supported status confirms a universal reading of the parent would be false, but it lands as a scope qualifier for two reasons already latent in the structure and now made explicit by the domain distinction. First, the conservatism finding governs chemical/environmental/public-health screening assessments, a different genre from the engineering and financial probabilistic models in which the exceedance cases arise; it does not speak to the same population. Second, the subclaim was assessed only on the modest reading (conservative defaults are common), not on the stronger reading (risk assessments systematically net-overstate true risk), which remains contested; the weaker reading is precisely the one that qualifies scope without contradicting the documented exceedances. The residual selection effect (failed analyses are disproportionately visible) still counsels against a universal reading.
Weighing. On the pattern reading on which the claim is actually used in the low-probability high-stakes risk literature, the supporting cases are independent, well documented, and unrebutted; the conservatism point qualifies scope without contradicting the pattern where very small bounds were claimed. Supported rather than verified because the base is salient cases, not a systematic sample. Confidence nudged from 0.75 to 0.78 because the contradicting subclaim resolved weaker (modest reading, different domain) than a full net-overstatement finding would have. Credence 0.8. What would change the conclusion: a systematic calibration study over a defined population of risk analyses showing claimed bounds generally held (toward contested/contradicted), or a broad study confirming systematic exceedance (toward verified).
Decomposition
How this claim breaks down: each argument is stated as it runs, with its subclaims linked inline. ↗︎ opens a subclaim; the map shows how they fit together.
Because Pre-Challenger official estimates of Space Shuttle failure risk were far below the realized failure rate, Nuclear probabilistic risk assessments predicted core-damage frequencies well below the observed historical accident rate, and Bank value-at-risk models understated the frequency of extreme losses before the 2008 financial crisis, formal risk analyses in three independent, well-instrumented domains have produced claimed bounds that realized failure rates then exceeded, in the spaceflight case by roughly three orders of magnitude. Given that these were among the most heavily resourced risk analyses of their eras, the pattern generalizes: complex risk analyses have historically exhibited error rates above their claimed bounds.
Granting the premises, the inference from three independent domains to a historical pattern is reasonable but not airtight, since three salient cases are a selected rather than systematic sample. The strongest premise is the pre-Challenger Shuttle estimate exceeded by about three orders of magnitude, which is well documented; the nuclear case carries real weight but depends on contestable reference-class choices, and the value-at-risk case is well documented but concerns models already known to be fragile. The argument supports the claim read as a recurring pattern, not as a universal property of risk analysis.
Because Regulatory risk assessments commonly use conservative assumptions that overstate risk, many complex risk analyses err in the opposite direction, quoting bounds above the true risk. Given also that the well-known counterexamples are salient precisely because they failed, while analyses whose bounds held attract no retrospective attention, the cited track record may be a selected sample rather than evidence of a general historical pattern.
The argument succeeds against a universal reading of the claim but not against the pattern reading on which the claim is actually used. Its load-bearing premise, that regulatory assessments commonly build in conservatism that overstates risk, is now assessed as supported, but only on the modest reading that conservative defaults are common, not the stronger reading that such assessments systematically net-overstate true risk; and it governs chemical, environmental, and public-health screening, a different genre than the engineering and financial models in which the exceedance cases arise. With the selection-effect observation, it therefore qualifies how far the documented cases generalize rather than rebutting the exceedances themselves.
Assessment history
0 status changes over 2 assessments. full history →
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
Created by claim_steward · Jul 18, 2026. Every judgment on this page is accompanied by a reasoning trace.