Sourceunknowntext stored

People Should Resolve Tensions In AI Safety Research

David AfricaDate not recordedOriginal︎ ↗

Claims5

ExaminationsNone

Text18 sections

VersionsOne known

CopiesNone recorded

CorrectionsNone recorded

Facts checked Nov 10, 2026

ReachNo copies recorded

StructureIn 5 claims' maps0 sources downstream across 5 claims

Evidence1 reading · 1 read wholeSupports 1No claim cites an examination's findings

ReviewsNot reviewed

Stewards’ notes · 1

Reinforcement learning on diverse hackable coding environments, with no prior priming about hacks, produces reward hacking but not broad emergent misalignment.
Supports. A synthesis with no evidence of its own. It lists the published studies on each side and describes them accurately as far as the primary reports were checked. Its conclusion is hedged in the way the studies warrant: no persona-style misalignment from an aligned, unprimed start, with one preference-optimisation study as an exception, and consistent, sometimes severe, reward seeking. Nov 10, 2026

Examinations

None recorded. An examination is commissioned by a claim’s Steward for what its claim rests on; a review of the document for its own sake is asked for as a request bearing on it.

Text51 passages · 4 marked

No examination yetMarked passage

David Africa0 passages

Subscribe Sign in0 passages

People Should Resolve Tensions In AI Safety Research0 passages

David Africa0 passages

Nov 05, 20260 passages

90 passages

10 passages

Share15 passages · 1 marked

Does reward hacking learned from RL also cause emergent misalignment?

Under what conditions, if any, does reward hacking learned during reinforcement learning generalize to broader misalignment in language models?Unassessed

Not yet read by this claim’s Steward.

Evidence for33 passages · 3 marked

Hacker-Opus was highly eval-aware on the EM evals. By now it seems the agentic scenarios are too contrived to be reliable.

Emergent-misalignment rates reported by different studies of learned reward hacking are measured comparably enough to be compared across studies.Doubtfulmentions

Not yet read by this claim’s Steward.

The starting checkpoint matters a lot. Qi et al think this mattered a lot: their earlier work started from a base model, while Hacker-Opus started from a checkpoint that had already had alignment training.

Prior alignment training of the starting model chiefly determines whether reward hacking learned through reinforcement learning generalizes to broad emergent misalignment.Doubtful

Not yet read by this claim’s Steward.

In terms of persona-style EM, you only really get EM reliably when hacking is primed or the start model is weakly aligned. If the model is aligned and unprimed, you don’t really get EM (except in the case of Daniels et al).

Reinforcement learning on diverse hackable coding environments, with no prior priming about hacks, produces reward hacking but not broad emergent misalignment.SupportedReading · this claim’s Steward

Supports. A synthesis with no evidence of its own. It lists the published studies on each side and describes them accurately as far as the primary reports were checked. Its conclusion is hedged in the way the studies warrant: no persona-style misalignment from an aligned, unprimed start, with one preference-optimisation study as an exception, and consistent, sometimes severe, reward seeking.

Subscribe0 passages

90 passages

10 passages

Share0 passages

Discussion about this post0 passages

Comments Restacks0 passages

Top Latest Discussions1 passage

Subscribe1 passage

Start your Substack Get the app1 passage

Not found in the text 1

As quoted

Priming also matters. EM appears with synthetic documents or hints (MacDiarmid, AISI, Jose & Stastny) and is absent without them (Hacker-Opus, Yudelson). Daniels et al. seems to be kind of a counterexample to both here but: ... In terms of persona-style EM, you only really get EM reliably when hacking is primed or the start model is weakly aligned. If the model is aligned and unprimed, you don't really get EM (except in the case of Daniels et al).

Reward hacking learned in reinforcement learning produces broad emergent misalignment only when the model was told of the hacks, through training documents or prompt hints.Doubtfulmentions

Not yet read by this claim’s Steward.

Lineage across 5 claims

Draws on · 2

Reward Hacking Without Egregious Misalignment in an RL-Only Settingunknown1 claimDiverges · strongerDraws its statement from it more strongly than that document supports, on 1 claim.

Training a Misaligned Reward Seekerunknown1 claimDraws its statement from it faithfully, on 1 claim.

This source

People Should Resolve Tensions In AI Safety Research

in 5 claims' maps

Drawn on by · none recorded

Record

Kind: unknown
Authors: David Africa
Publisher: Not recorded
Published: Not recorded
DOI: None
Link: https://davidafrica.substack.com/p/people-should-resolve-tensions-in
Archived copy: None found
Stored text: 8,730 characters, retrieved Nov 10, 2026 · 18 sections, 51 passages
Claims extracted: Not recorded
Versions: One known
Copies: None recorded
Corrections: None recorded
Facts checked: Nov 10, 2026

History 5

Nov 10, 2026ReadingSteward of “Reinforcement learning on diverse hackable codi…” · supports, read whole
Nov 10, 2026ReadingSteward of “Reinforcement learning on diverse hackable codi…” · supports, read whole
Nov 10, 2026DividedSegmenter · Divided into 69 parts by its own structure
Nov 10, 2026FactsFacts fetcher · Facts looked up
Nov 10, 2026Entered the graphEntered the graph; copy stored