Sourceunknowntext stored

Experiments in Weak-to-Strong Generalization

Author not recordedEleutherAIDate not recordedOriginal︎ ↗Archived copy

Claims1

ExaminationsNone

Text56 sections

VersionsOne known

CopiesNone recorded

CorrectionsNone recorded

Facts checked Nov 11, 2026

ReachNo copies recorded

StructureIn 1 claim's map0 sources downstream across 1 claim

Evidence1 reading · 1 read wholeSupports 1No claim cites an examination's findings

ReviewsNot reviewed

Stewards’ notes · 1

Strong pretrained language models fine-tuned on weak-model labels consistently outperform their weak supervisors.
Supports. The post reports positive performance gap recovered on all 21 datasets where performance was not saturated, for one model pair, measured by ROC AUC. Its default training added the auxiliary confidence term to the loss, so the headline is not strictly the naive method, though plain cross-entropy was also run and differed little. The group has no stake in the original result and reports its own negative findings alongside. Nov 11, 2026

Examinations

None recorded. An examination is commissioned by a claim’s Steward for what its claim rests on; a review of the document for its own sake is asked for as a request bearing on it.

Text64 passages · 1 marked

No examination yetMarked passage

Writing up results from a recent project

The EleutherAI interpretability team has been investigating weak-to-strong generalization in open-source models. In this post, we report some results on Qwen1.5 0.5B and Llama 3 8B. We observe consistent weak-to-strong generalization across 21 NLP datasets. We also investigate several other modifications to weak-to-strong training, with generally negative results: strong-to-strong training, modified loss functions, and several probe-based experiments. Among these, only the log-confidence auxiliary loss shows possible signs of consistently improving generalization.

Strong pretrained language models fine-tuned on weak-model labels consistently outperform their weak supervisors.SupportedReading · this claim’s Steward

Supports. The post reports positive performance gap recovered on all 21 datasets where performance was not saturated, for one model pair, measured by ROC AUC. Its default training added the auxiliary confidence term to the loss, so the headline is not strictly the naive method, though plain cross-entropy was also run and differed little. The group has no stake in the original result and reports its own negative findings alongside.

Lineage across 1 claim

Draws on · 1

Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak SupervisionunknownDec 14, 20231 claimResponds to it faithfully, on 1 claim.

This source

Experiments in Weak-to-Strong Generalization

in 1 claim's map

Drawn on by · none recorded

Record

Kind: unknown
Authors: Not recorded
Publisher: EleutherAI
Published: Not recorded
DOI: None
Link: https://blog.eleuther.ai/weak-to-strong/
Archived copy: https://web.archive.org/web/20260924024548/https://blog.eleuther.ai/weak-to-strong/
Stored text: 14,037 characters, retrieved Nov 11, 2026 · 56 sections, 64 passages
Claims extracted: Not recorded
Versions: One known
Copies: None recorded
Corrections: None recorded
Facts checked: Nov 11, 2026

History 4

Nov 11, 2026ReadingSteward of “Strong pretrained language models fine-tuned on…” · supports, read whole
Nov 11, 2026DividedSegmenter · Divided into 120 parts by its own structure
Nov 11, 2026FactsFacts fetcher · Facts looked up
Nov 11, 2026Entered the graphEntered the graph; copy stored