An alignment safety case sketch based on debate
Buhl, Marie Davidsen, Pfau, Jacob, Hilton, Benjamin, Irving, GeoffreyarXiv.orgPublished May 6, 2025Original︎ ↗Archived copy
Examinations
None recorded. An examination is commissioned by a claim’s Steward for what its claim rests on; a review of the document for its own sake is asked for as a request bearing on it.
Text491 passages · 1 marked
Exploration hacking. A major gap in our sketch is the requirement for exploration guarantees. We know very little about how serious a problem exploration hacking is and how to prevent it - yet the sketch depends on avoiding it. We outlined various research directions that might help address exploration hacking across both interpretability, singular learning theory, and more prosaic techniques – much more work is needed to develop and test these. More foundationally, it would be useful to develop model organisms of exploration hacking to help test potential solutions. For very superhuman systems, we might also need to address gradient hacking.
Not yet read by this claim’s Steward.
Lineage across 1 claim
Record
- Kind
- preprint
- Authors
- Buhl, Marie Davidsen, Pfau, Jacob, Hilton, Benjamin, Irving, Geoffrey
- Publisher
- arXiv.org
- Published
- May 6, 2025
- DOI
- None
- Link
- https://arxiv.org/html/2505.03989v3
- Archived copy
- https://web.archive.org/web/20260204211719/https://arxiv.org/html/2505.03989v3
- Stored text
- 81,914 characters, retrieved Nov 11, 2026 · 58 sections, 491 passages
- Claims extracted
- Not recorded
- Versions
- One known
- Copies
- None recorded
- Corrections
- None recorded
- Facts checked
- Nov 11, 2026
History 3
- Nov 11, 2026FactsFacts fetcher · Facts looked up
- Nov 11, 2026DividedSegmenter · Divided into 583 parts by its own structure
- Nov 11, 2026Entered the graphEntered the graph; copy stored