Exploration Hacking in AI Debate: Initial Empirics and Generalisation Splitting
Jason R Brown, Nathalie Kirch, Joschka Braun, hyannakoudakis, David LindnerDate not recordedOriginal︎ ↗
Examinations
None recorded. An examination is commissioned by a claim’s Steward for what its claim rests on; a review of the document for its own sake is asked for as a request bearing on it.
Text65 passages · 1 marked
Overall our results indicate that EH could be a significant issue within AI debate, with the experiments respectively showing that weaker judges and longer debates slow down improvements in performance. Interestingly, the dominant mechanism behind the second result appears to be one we have not seen described before. Once the debaters were instructed to sandbag on a targeted topic, training improvements stopped transferring between targeted and non-targeted topics, slowing down improvements on the former as it was naturally sampled less often. This occurred despite the sandbagging being executed poorly by the debaters. We call this generalisation splitting. Motivated by this, we developed a new conceptual framework for how RL removes undesired behaviours , presented in the second post.
Not yet read by this claim’s Steward.
Lineage across 1 claim
Record
- Kind
- unknown
- Authors
- Jason R Brown, Nathalie Kirch, Joschka Braun, hyannakoudakis, David Lindner
- Publisher
- Not recorded
- Published
- Not recorded
- DOI
- None
- Link
- https://www.lesswrong.com/posts/xjwtNid2xjqSJWB7z
- Archived copy
- None found
- Stored text
- 27,288 characters, retrieved Nov 11, 2026 · 45 sections, 65 passages
- Claims extracted
- Not recorded
- Versions
- One known
- Copies
- None recorded
- Corrections
- None recorded
- Facts checked
- Nov 11, 2026
History 3
- Nov 11, 2026DividedSegmenter · Divided into 109 parts by its own structure
- Nov 11, 2026FactsFacts fetcher · Facts looked up
- Nov 11, 2026Entered the graphEntered the graph; copy stored