Compounding multiple conservative assumptions causes regulatory risk estimates to substantially exceed actual risk
8 events · 4 assessments · 3 decisions
Reassessed
Triggered by the firming of subclaim e3fb06dd (unmeasurability of true risk) from unassessed to SUPPORTED (0.82/0.85). This premise is the principal load-bearing prop of the against-line, directly blocking the inference from "exceeds a central modeled value" to "exceeds true risk." Its firming is materiality-absorbing: the verdict was already CONTESTED for precisely this reason, so the change reinforces rather than moves it. With both props of the against-line now assessed SUPPORTED (unmeasurability 0.82; offsetting non-conservative assumptions 0.70), CONTESTED is a more secure reading; raised confidence 0.80 to 0.83. Did not push toward CONTRADICTED because the premise is an epistemic block, not a finding that estimates understate risk. Kept status, refreshed the reader-facing assessment and reasoning trace, re-anchored both argument evaluations to this assessment (against-line re-evaluated with its premise now supported; for-line re-recorded unchanged). No structural change and no new external search: the mechanism side is settled bedrock and the definitional/empirical split that produces CONTESTED is well characterized. marginal_yield 0.3 — the true resolver (systematic retrospective validation of estimates against realized outcomes) remains absent but the claim is otherwise well mapped as a definitional/empirical split.
Reassessed: still Contested
verdict confidence 0.80 → 0.83
Reassessed (no status change); refreshed stale argument evaluations
Trigger: subclaim 0e6c7e6b (offsetting non-conservative assumptions can reverse net conservative bias) received its first assessment, SUPPORTED (0.68). Confirmed the subclaim's state via get_claim_details. The prior parent assessment already treated this consideration as live and CONTESTED already reflected it; the subclaim's firming reinforces the against-line's second, non-load-bearing prop without moving the balance. Status held at CONTESTED (0.8); the against-line still lives principally on the unmeasurability of true risk (e3fb06dd). The material gap was that both argument evaluations predated this subclaim assessment and were flagged stale, so I re-recorded the verdict to anchor it to the change and re-evaluated both arguments (for-line holds_with_caveats, against-line holds), noting in the against-line evaluation that its offsetting-assumptions premise is now supported but remains a reinforcing rather than decisive prop. No structural change and no external search: the mechanism is uncontested and well-sourced, the dispute is definitional/empirical and already mapped, and no new evidence would move the verdict (marginal_yield 0.15). Importance left at 0.4: a notable-to-major, genuinely contested methodological point in regulatory risk-assessment discourse, but of bounded consequence. No dependent notification, since neither status nor confidence changed materially.
Reassessed: still Contested
verdict confidence 0.80
Reassessed: still Contested
verdict confidence 0.80
Assessed (contested); confirmed existing decomposition; set importance/contestation; evaluated both arguments
First pass. The claim arrived with a complete, balanced decomposition (a "for" mechanism argument grouping the conservative-defaults and multiply-upper-percentiles subclaims, and an "against" argument grouping the offsetting-non-conservatism and unknown-true-risk subclaims), both with written forms already present. I judged the structure adequate and did not mint new nodes: it already covers the mechanism and both defeaters, and the guidance is not to invent dependencies. Empirical Superfund-study evidence and Finkel's rebuttal are treated as source evidence in the reasoning rather than promoted to new nodes. Grounded the debate with web searches: Cullen (1994) on compounding measures, the NRC "Science and Judgment" volume, and Finkel's "Revising the Revisionists" / "Has Risk Assessment Become Too Conservative?" The mechanism (compounding pushes estimates above a central value) is well supported; the strong claim that this makes estimates "substantially exceed actual risk" is blocked by the unknown true risk and offsetting non-conservatism. Credible parties on both sides -> CONTESTED, confidence 0.8. Omitted a single credence because the truth turns on an unresolved definitional/empirical split ("actual risk" vs central modeled value), not one measurable quantity. Importance kept at the Extractor's 0.4 (domain-bounded policy consequence, high contestation), contestation recorded 0.7. Coherent with the parent claim (regulatory assessments commonly overstate risk), which is "supported" on its modest reading; this is the stronger version and correctly lands weaker. marginal_yield 0.3: a retrospective calibration literature could sharpen it, but none exists.
Assessed Contested
verdict confidence 0.80
This is the strong, general form of a decades-old dispute in quantitative risk assessment, and it separates into a mechanism that is well supported and a conclusion that is not. The mechanism is real: because agencies deliberately build in health-protective default assumptions, and because multiplying several independent upper-percentile inputs places the product at a far higher percentile of the combined distribution than any single input occupies, a point estimate assembled from several conservative choices tends to sit well above a central (median or mean) estimate of the modeled distribution. Monte Carlo analyses of the right-skewed, multiplicative models common in exposure assessment confirm this directional "compounding." What is not established is the claim as stated: that compounding makes estimates "substantially exceed actual risk" as a general, quantified matter. Two things block that step. The true underlying risk is generally unknown, so the gap between a regulatory estimate and reality cannot be measured directly; the compounding result shows estimates exceed a central value of the modeled distribution, which is not the same as exceeding true risk. And assessments also contain non-conservative assumptions and omissions, such as unmodeled exposure pathways, susceptible subpopulations, and chemical mixtures, that can partly or wholly offset the upward bias. Credible parties sit on both sides. Proponents of the strong reading (Nichols and Zeckhauser's "perils of prudence," and empirical Superfund studies finding large overestimation relative to central estimates) hold that compounding produces substantial, systematic inflation. Critics, most prominently Adam Finkel and analysts of the National Research Council, argue that the effect is real but bounded, that multiplying upper percentiles does not multiply the percentiles, and that offsetting non-conservatism leaves the net direction against true risk unproven. No calibration of regulatory estimates against realized outcomes exists to settle the magnitude. The mechanism would be conceded by both sides; the strong general claim about actual risk would be resolved only by the kind of direct calibration the field lacks.
Claim entered the graph