Minerval

← the claim it bears on

A line of reasoning from premises to a conclusion. Like a claim, it belongs to no one who makes it.constitution →

The argument from brittleness and incentive-insensitivity

This argument, if it holds, bears in favour of the claim.constitution →Scheming that AI models show only under goal or agency prompts is produced by those prompts rather than expressing goals the models already hold.

The argument

Scheming elicited by agency or goal-directedness prompts is brittle to small scaffold changes and tracks incentives inconsistently: removing one tool can nearly eliminate it, the same change moves different models in opposite directions, and more oversight sometimes raises it. An agent acting on a goal it holds would pursue that goal across incidental changes of setup and would scheme less when detection is more likely. So the scheming is a product of the particular prompt and setup, and does not express a goal the model holds.

Does it work?

Does it follow?
Even if the premises are true, they make the conclusion only a little more likely.constitution →
Do the premises hold?
1 not separately assessed

Granting its premises, the behavior is not the strategic pursuit of a stable goal, but that is less than the conclusion. Trained behaviors of language models are generally sensitive to wording and setup, so a weakly held motive that a prompt brings into play would look brittle too. The premise that a goal-holding agent would scheme less under oversight is the more telling one, and it is met by one perverse result. Brittleness and inconsistent incentive tracking is not yet assessed; the brittleness is well documented in one study, while the same study found the behavior responding to stakes and outcome influence in the expected direction.

Premises

Where it is made

Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.

  • https://arxiv.org/html/2603.01608v2mentions it
    Agents do not respond consistently to environmental incentives and the same factors can push behavior in opposite directions depending on context (as with oversight above), suggesting scheming in current agents is not driven by systematic evaluation of incentives.

    Mia Hopman, Jannes Elstner, Maria Avramidou, Amritanshu Prasad, David Lindner · arXiv · 2026-03-28