Reasoning in LLMs: Chain of Thought and Long-Reasoning RLVR
Chapter 18 separates two levers for LLM reasoning: chain of thought adds test-time computation, while verifiable rewards and policy gradients train long-reasoning behavior.
#ppo
2 postsPolicy Gradient and Its Variants: REINFORCE and PPO
Chapter 21 derives REINFORCE with the log-derivative trick, then uses reward-to-go, baselines, and PPO clipping to control policy-gradient variance and update size.