CMU 07-280 Lecture 22: Q-learning When Dynamics Are Unknown
Lecture 22 keeps the MDP structure but removes known transitions and rewards. TD learning updates value from one sample, and Q-learning uses an off-policy target to learn optimal action values directly.