CS224R L13: Meta-RL, Teaching an Agent to Learn New Tasks Fast
Meta-RL trains on many tasks so that a new task can be solved from a small amount of experience. Lecture 13 of CS224R frames it as "explore to collect a little data, then adapt using that data." The most direct approach is black-box meta-RL (RL²): a network with memory takes past (s, a, r) as input and keeps its hidden state across episodes. It is general and expressive but hard to optimize, especially when exploration is hard, because exploration and execution depend on each other and end-to-end training gets stuck. The slides then compare posterior sampling in PEARL, prediction-driven exploration in MetaCURE, and DREAM, which uses a task representation to train exploration and execution separately.