Step 6: tool results

{
  "content": "On five algorithmic tasks (permutation powering, sequential assignments, wildcard search, shortest path, union find), supervised learning needed tens of millions of full-solution examples while Iterated Amplification needed tens of thousands of examples of one-step decompositions. The tasks share the feature that each step of decomposition is far simpler than the whole computation. If realistic tasks share that feature to a similar degree, the supervision savings should carry over to them.",
  "argument_id": "aabfd608-54bf-4a9e-a901-87a3c94c8108"
}

{
  "content": "Producing a decomposition while able to call a solver for the subtasks is strictly easier than solving the task without that help. A behavior that is easier to perform is easier to learn, and so needs fewer supervised examples. Therefore learning to decompose a realistic task should require less supervision than learning to perform it directly.",
  "argument_id": "920417c3-d150-4426-bfe8-9fb9add56d4c"
}

{
  "content": "The toy tasks were chosen because they decompose cleanly into small, explicit steps. Human experiments on factored cognition with realistic material (Ought's work on reading comprehension and argument evaluation, 2019 to 2020) found that decomposing such tasks into locally checkable pieces was hard, slow and error-prone, and that participants disagreed about whether steps were valid. If realistic tasks resist clean decomposition, demonstrations of good decompositions are expensive and the supervision saving seen on toy tasks need not appear.",
  "argument_id": "26705e2a-1bae-4ef2-af33-3eb5852f67b2"
}