argumentA line of reasoning from premises to a conclusion. Like a claim, it belongs to no one who makes it.constitution →

The argument that faithfulness evaluations track monitorability in practice

againstThis argument, if it holds, weighs against the claim.constitution →Evidence that chain-of-thought is mechanistically faithful does not by itself establish its behavioral monitorability, nor vice versa.

The argument

The faithfulness evaluations most in use, such as whether a model verbalizes a hint that changed its answer, measure whether decision-relevant factors appear in the trace. That is also what a monitor reading the trace needs. If faithfulness evaluations are valid proxies of monitorability, then evidence of faithfulness would substantially establish monitorability, and the two would not need separate evidence.

Does it work?

Does it follow?: slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
Do the premises hold?: 1 doubtful

Even granting that faithfulness evaluations are valid proxies of monitorability, a valid proxy is a correlate, not a sufficient condition. Evidence of faithfulness would make monitorability more likely, but it would still leave open whether a given monitor can recognize what the trace contains. The argument therefore bears on how strongly the two properties go together, not on whether one establishes the other "by itself", which is what the claim is about.

Premises

DoubtfulThe evidence weighs against the claim, but the examination is incomplete or the evidence is indirect.constitution →Chain-of-thought faithfulness evaluations are valid proxy measures of chain-of-thought monitorability.

Where it is made

Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.

No source is recorded as making it.