Skip to content
All tags

#deepseek-r1

2 posts
aideep-diveReading Stanford CME295 Transformers & Large Language Models

How Reasoning Models Learn to Think Longer, and What GRPO Drops from PPO | Stanford CME295 Lecture 6

CME295 Lecture 6 breaks reasoning models into three pieces: emit a reasoning chain before the answer, run RL on verifiable rewards like "is the answer correct," and use GRPO, which takes the group's average reward as the baseline instead of training a value model. RL alone took DeepSeek-R1-Zero from 15.6% to 71.0% pass@1 on AIME 2024, and distilling R1's traces into Qwen-32B beat running RL on the 32B model directly.

#cme295#reasoning#grpo#rlvr#deepseek-r1#stanford
aideep-diveReading Stanford CS224N Natural Language Processing with Deep Learning

Decoding, DeepSeek-R1, and Reasoning Training | Stanford CS224N Lecture 12

Lecture 12 shows that output policy is not a detail: greedy, beam, and sampling produce different text. It then moves from R1-Zero/R1 into PPO, GRPO, and DAPO, asking when longer reasoning actually helps.

#cs224n#reasoning#deepseek-r1#reinforcement-learning#stanford