Hung-yi Lee opens his May 8 lecture by admitting that "self-improving AI" has no clear definition: it is a process of humans gradually letting go. He splits machine learning into three steps and checks where the "I" can be replaced by AI. Answers can come from the model's own self-corrections, reward shaping can be written by an LLM, the loss can be set by the model itself (scores, majority vote, entropy), and even the questions can come from a proposer model. But experiments keep showing that with no human at all, progress plateaus or the model trains itself into the ground. A strong AI can already train a weaker one, just not better than humans do. His verdict: in May 2026, AI is "still standing at the bank of the Rubicon."
Part 1 was about an AI setting its own loss and updating its own parameters. Part 2 fills in the other half: AI Agent = Harness + LLM, and the harness can grow too. You can't take a gradient through a harness, so the usual move is to hand it to a language model as a rewriter and keep a pool of candidates, much like a genetic algorithm (OPRO, GEPA, Darwin Gödel Machine; DSPy if you want a ready-made tool). Three extensions follow: updating harness and parameters together beats updating either alone; when the goal changes you have to choose between discarding everything and carrying everything, and editing a harness can cause forgetting too; and the update rule itself can be updated (HyperAgent, Gödel Agent, SEAL), which is meta learning. Hung-yi Lee closes with a new analogy — parameters are genes, context is the neurons — then argues that today's agents lack intrinsic motivation, and that the likeliest source of runaway growth is a gap between the goal humans meant and the goal the AI inferred.
When a research agent runs 25, 100, or 2000 turns, what happens? Context suffocation: information piles up, noise increases, attention gets diluted. IterResearch solves this with Markovian state reconstruction; AREX achieves recursive self-improvement with an inner/outer loop. Both answer: how does an agent stay coherent across hundreds of search rounds?
Three non-big-lab teams used different RL post-training strategies to produce benchmark dark horses in 2026: Ornith's self-improvement loop (GRPO), Nous Research's DataForge + Atropos execution-reward RL, and MiniMax's massive-scale RL across 200K real environments. Different strengths, but one shared proof point: post-training RL matters more than pretraining scale.
Hermes memory has hard caps: 2,200 characters for MEMORY.md and 1,375 for USER.md, and an over-limit write returns an error instead of auto-compacting, forcing the agent to make room itself. Skills are maintained by a curator that runs every 7 days after 2 hours of idle, marks skills stale at 30 days and archives at 90 — but never deletes. The switches actually worth flipping are `memory.write_approval` and `skills.write_approval`, which stage the background self-improvement writes for review.