Evidence assembled in a safety argument retains value even if the argument's reasoning is flawed
2 events · 1 assessment
Assessed Supported
verdict confidence 0.75 · credence 0.85
A safety case bundles two distinguishable things: items of evidence such as test results, design analyses, and operational history, and an argument connecting them to a conclusion about risk. The claim holds that when the argument fails, the evidence does not fail with it. This rests on the premise that operational and test data bear on system risk independently of the arguments that cite them, a position the assurance-case literature broadly shares: standard treatments of confidence in assurance cases assess trust in the evidence and trust in the reasoning as separate contributors, so a defect in one does not automatically zero out the other. A thousand hours of failure-free operation remain a thousand hours of failure-free operation whatever inferential use was made of them. Two credible qualifications limit how much value is retained, without overturning the claim. First, critics of the safety-case regime argue that the evidence in such cases is selected under confirmation bias to support a predetermined conclusion, and that the flaws that actually defeat safety assurance are principally missed hazards, in which case the assembled evidence may address the wrong risks entirely. Second, feasible amounts of testing cannot by themselves demonstrate the ultra-low failure rates safety cases claim, so evidence stripped of its connecting argument supports far weaker conclusions than the original case did. Read strictly, the claim asserts only that some evidential value survives, which these qualifications discount but do not eliminate; read as asserting that most of the case's assurance survives a flawed argument, it would be contested. Empirical study of what discovered flaws in real safety cases did to the underlying evidence's relevance would sharpen the answer.
Claim entered the graph