A chat model is rewarded for a single response people prefer; an agent has to be rewarded for whether a multi-step task actually finished, which needs different data and environments.
Reasoning is the intermediate tokens between input and output; RL fine-tuning beats imitating human solutions because of a reliable verifier, and sampling several times then taking the most frequent answer pushes accuracy further.
AlphaGeometry2 relies on a symbolic engine and AlphaProof on Lean, for a combined 28 points in 2024; in 2025 Gemini Deep Think scored 35 points in natural language, with verification done by human graders.
Laskin's view is that engineers spend 80% of their time understanding systems rather than writing code, and that how far RL goes depends on clean rewards, a problem he considers as hard as superintelligence itself.
A VLA adds an action module to a vision-language model; letting that module's gradients flow back harms language understanding, so knowledge insulation blocks the gradient, while π0.5 co-trains on web data and data from many robots to gain generalization.
An agent needs trustworthy feedback to correct its actions; improving one task’s result does not mean its model weights have learned.
Open-ended evolution changes agent code and workflows; archives preserve branches, while held-out tests assess generalization.
Program candidates need requirement-based counterexamples; speed matters only after correctness checks pass.
Retrieval, trained KV representations, and cache reuse address different bottlenecks; speed does not guarantee freshness.
More candidates increase the chance of a correct solution; credible selection turns that chance into delivered success.
A correct final answer does not validate its derivation; step feedback locates errors, but verifiers still need checking.
Observations, tests, and principle-based critique supply different signals; identify the signal before choosing revision or training.
Distinguish search, parallel execution, and training; subtask artifacts must fulfill the original task criterion.
Useful training feedback must distinguish candidates; filtering, relative advantages, and sampling determine the learning signal.
Candidate search selects solutions; retrieval fills knowledge gaps. Both require specifications and checkable evidence.
Human task duration, artifact quality, and citation support measure different things; evaluate both execution and deliverables.
Diversity, verification, task selection, and efficiency are distinct bottlenecks; experiments need holdouts and falsifiable hypotheses.
Generating a correct candidate and recognizing it are different capabilities. This series follows the Autumn 2025 syllabus through a course overview and seventeen session guides covering all seventeen content sessions; five guest sessions have no public material and use public material from the speakers on similar topics instead; video Parts 7–9 map to sessions 8, 17 and 20.