argumentA line of reasoning from premises to a conclusion. Like a claim, it belongs to no one who makes it.constitution →

The argument from discovery case studies

forThis argument, if it holds, bears in favour of the claim.constitution →Sparse autoencoders are powerful tools for discovering previously unknown concepts represented in language models.

The argument

A sparse dictionary enumerates many interpretable features without being told what to look for, and a researcher can then select those that satisfy a criterion and test them by other means. Used this way, sparse dictionaries have surfaced mechanisms that researchers had not specified in advance, such as a model choosing its rhyme word before writing the line, and in one audit SAE tools raised an automated agent's success at finding a hidden objective. Methods that need a concept specified in advance, such as probes and steering vectors, cannot carry out this enumeration. So SAEs are powerful tools for discovering unknown concepts in language models.

Does it work?

Does it follow?: slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
Do the premises hold?: 2 not separately assessed

Granting its premises, the argument shows that sparse dictionaries can surface concepts researchers had not specified, but "powerful" needs more: that they find such concepts more often or more cheaply than alternatives. Case studies without comparators cannot show that, and the premise that only dictionaries can enumerate candidate concepts is not established, since behavioural elicitation, embedding search and language-model summarisers also generate unprompted hypotheses. The argument rests on linked claim, a well-evidenced finding with no matched alternative, and on linked claim, a positive result conditional on the dictionary's training data. Not decisive.

Premises

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →Claude 3.5 Haiku represents a couplet's intended rhyme word before generating its second line.
UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →In Anthropic's reward-model-sycophancy audit, sparse-autoencoder tools improve automated agents' objective-discovery success over chat and training-data search tools alone.

Where it is made

Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.

Position: Use Sparse Autoencoders to Discover Unknownsmakes itleaves a premise unstatedThe passage relies on a premise the inference needs without saying it.constitution →
In both cases, our hope is to discover concepts that satisfy a property of inter- est, out of intractably many possibilities. SAEs produce a set of concepts that is both tractable and expressive, after which selecting a subset of concepts that are relevant to the task is straightforward.