Skip to content
All tags

#value-iteration

2 posts
aiguideReading Stanford CS234 Reinforcement Learning

Planning with a Model: Policy Evaluation, PI, VI | Stanford CS234 Lecture 2

Lecture 2 of CS234 Winter 2026 assumes the world model is known and asks how to compute the best policy. An MDP plus a policy is an MRP, so a policy can be evaluated by iterating a Bellman backup. Policy iteration alternates evaluation and improvement, and the slides prove each round is no worse than the last, so it stops within |A|^|S| rounds. Value iteration takes another route: apply the Bellman optimality operator over and over. For γ < 1 that operator is a contraction, so value iteration always converges. The lecture ends with finite horizons, where the best policy usually depends on how many steps remain.

#cs234#ai-course#stanford#reinforcement-learning#mdp#value-iteration#dynamic-programming
techguideReading Harvard CS1810 Machine Learning

(Part 3): Policy Iteration and Value Iteration for MDPs | Harvard CS181 Assignment 6

HW6 Problem 2 (15 pts) hands you a 4×5 Gridworld where moves can slip and rewards arrive only when you leave a cell. You write one step each of policy evaluation, policy iteration, and value iteration in the notebook, watch how the discount factor γ reshapes the policy, and finally ask whether this is a sensible model of the robot task at all. The rules of the world are fully known, so this is planning, not learning yet.

#harvard#cs181#mdp#value-iteration#dynamic-programming#reinforcement-learning#homework