Reinforcement Learning: MDPs, Value Iteration, and Continuous States
Chapter 19 uses Bellman equations to turn long-horizon decisions into one-step updates, moving from value iteration in known MDPs to model learning and continuous-state approximation.