Table of Contents
🌏 中文版
The official schedule starts with Introduction, Behavioral Cloning, Behavioral Cloning Part 2, and RL Basics. The point is not to memorize an RL algorithm first. It is to see where a supervised controller fails, then introduce learning from reward.
L1–2: control as supervised learning
Behavioral cloning trains a policy on expert state-action pairs. Its training loss is simple; deployment is not. Once the learned policy makes a small error, it may visit states absent from expert data. Draw the training distribution beside the distribution induced by the learned policy before naming the problem “covariate shift.”
L3 and Sections 1–2: make failure visible
Section 1 supplies a PyTorch tutorial, Section 2.1 reviews probability, and Section 2.2 focuses on BC distribution shift. HW1 compares an MSE policy, DAgger, and a flow-matching policy. DAgger asks the expert to label states actually visited by the learner, iteratively repairing the dataset.
L4: when RL becomes necessary
RL Basics reframes the task as an MDP. A policy produces a trajectory, rewards accumulate, and transition dynamics make today's action alter tomorrow's state. Expert actions permit direct imitation; outcome-only feedback introduces credit assignment and exploration.
HW1 and compute
The Spring 2026 starter code uses uv and Weights & Biases. This assignment is a sensible place to start on a local CPU; see the homework compute ledger for the supporting details. Retain three artifacts: a reward curve, generated behavior video, and a qualitative comparison of MSE, DAgger, and flow matching.
Public code is enough to implement the work, but it is not the complete enrolled experience. The series overview owns the full access boundary.
References
Loading...