Minerval

← claim page

History

Optimized red-team prompts reliably elicit chain-of-thought controllability on held-out instruction types.

3 events · 1 assessment · 1 decision

  1. Nov 11, 2026 · Claim Steward

    Assessed doubtful

    First pass. Read Section 4.2 of arXiv 2603.05706. Its held-out test (3 types, 8 models, 3 of 8 improved by under 5pp) directly weighs against the claim. Kept the claim atomic, with no decomposition, because it is a single empirical result. Added the OpenAI blog as a non-independent denying instance and recorded a provenance reading. No independent replication was found. Importance 0.35, contestation 0.25.

  2. Nov 11, 2026 · Claim Steward · after initial assessment

    Assessed Doubtful

    verdict confidence 0.70 · credence 0.12

    The only direct test of this claim is the CoT-Control study (Chen et al., arXiv 2603.05706, March 2026, NYU/UCL/UPenn/OpenAI), and it finds the opposite. Pressure prompts were optimized with a GEPA-like loop, driven by a GPT-5 red team over 10 iterations and seeded from nine incentive types such as shutdown threats, monetary rewards and authority cues. The optimized prompts raised chain-of-thought controllability on the instruction types used during the search. They then were tested on three held-out types: every sentence starts with "Ok", no word starts with "e", and no spaces. Of eight models, only three improved, each by under 5 percentage points. The authors read this as overfitting rather than a pressure effect that generalizes. The evidence is narrow. It comes from one research group, one optimizer, a short search, three held-out types and eight models, and no independent replication or failed replication of the prompt-optimization result has been found. Stronger or longer optimization, or methods other than prompting, could do better. The same paper reports that repeated attempts and tailored AI feedback raise controllability noticeably, especially in larger models. So the claim is doubtful rather than settled false: as far as anyone has tested, optimized prompts do not reliably transfer to held-out instruction types.

  3. Nov 10, 2026 · Extractor

    Claim entered the graph