One post per lecture of Stanford CS229’s public Spring 2026 recordings: from linear regression, neural networks, and EM to diffusion models, LLMs, and reinforcement learning for reasoning models. Seventeen lectures, each with the full derivations and the matching chapters of the official notes.
CS229 is the Stanford course that works through the mathematical derivations of machine learning. The Spring 2026 recordings make 17 lectures public: the first ten cover linear models, neural networks, and unsupervised learning, and the last seven go on to diffusion models, LLMs, and training reasoning models with reinforcement learning. This series has one post per lecture, and this post is the map: what each lecture actually teaches, where the video titles don't match, and where to pick up what the lecture notes cover but the lectures skipped.
The opening lecture of CS229 Spring 2026 runs only 37 minutes and has no derivations. It defines machine learning with Tom Mitchell's experience E, task T, and performance measure P, then splits the whole course into supervised, unsupervised, and reinforcement learning. The three differ in whether the data comes with answers and whether the model collected the data itself. Large language models show up in all three categories at once.
Lecture 2 breaks supervised learning into three pieces: a hypothesis h, the least-squares cost J(θ), and an algorithm that drives J down. Gradient descent sweeps all n examples on every step; minibatch SGD updates after looking at a small batch, and pays for it by bouncing around near the optimum. Linear regression also has a closed-form solution, θ = (XᵀX)⁻¹Xᵀy, provided XᵀX is invertible.
Assume the errors are zero-mean, IID Gaussian noise, take the log of the likelihood, and maximum likelihood turns out to be last lecture's least squares. Run the same recipe with 0/1 labels and a sigmoid and you get logistic regression, whose gradient is still "error times input". The lecture closes by comparing Newton's method with SGD: far fewer steps, but each step costs n·d² + d³.
If the distribution of y can be written as b(y)·exp(ηᵀT(y) − a(η)), one derivative of a(η) gives the expectation and two give the variance. Set η = θᵀx, and linear regression, logistic regression, and softmax all share one update rule: θ ← θ + α(y − h(x))x. The second half of the lecture uses a four-class example to cover softmax, cross entropy, and label smoothing.
Lecture 5 turns classification around: instead of learning p(y|x) directly, it learns p(x|y) and p(y). GDA assumes the two classes are Gaussians with a shared covariance. Maximum likelihood then takes nothing more than counting proportions and averaging, the decision boundary comes out as a straight line, and the posterior is exactly a sigmoid. Naive Bayes uses conditional independence to squeeze 2^d parameters down to 2d+1.
This lecture splits test error into three terms: noise σ², squared bias, and variance. It then uses the closed-form solution of ridge regression to show how regularization moves each eigenvalue from λ to λ+ρ, trading a little bias for a large cut in variance. The second half covers double descent, ImageNet-V2, and choosing hyperparameters with a dev set, k-fold cross-validation, and successive halving.
Lecture 7 first carries the training recipe for linear models over to nonlinear models unchanged: one loss per example, take the average, walk downhill with SGD. Two questions remain: what the model looks like, and how to compute the gradient. The second half answers the first. One neuron is ReLU(wᵀx+b); stacking m neurons into a matrix gives one layer, ReLU(Wx+b), with md+m parameters; above that come the residual block z+σ(MM(σ(MM(z)))) and LayerNorm and RMSNorm. Computing the gradient is left to Lecture 8.
The lecture starts from one theorem: if a scalar function can be computed with N basic operations, its gradient can also be computed in O(N) time. The method is to read the chain rule as "given ∂J/∂u, you can compute ∂J/∂z, using only information about the module in between", then derive a backward function for matrix multiplication and for the activation function: Wᵀv, vzᵀ, and σ′(z)⊙v. This article also works a forward and backward pass by hand on a small 2-2-1 network.
Lecture 9 starts from the two steps of k-means, assign and average, and explains why the algorithm always stops yet only reaches a local optimum. Replacing hard assignments with probabilities gives the Gaussian mixture model and the two steps of EM: an E-step that is Bayes' rule and an M-step that is a weighted average. The lecture ends with the picture behind EM, drawn with Jensen's inequality: each round builds a lower bound that touches the log-likelihood and climbs to its top. The full derivation waits for Lecture 10.
The first half uses Jensen's inequality to build a lower bound on the log-likelihood, the ELBO, then picks Q to be the posterior so the bound is tight at the current parameters. From that it follows that no round of EM can lower the likelihood, and plugging in a Gaussian mixture recovers last lecture's weighted-average formulas. The second half writes PCA as maximizing uᵀΣu over unit vectors; the answer is the eigenvector of the covariance matrix with the largest eigenvalue.
Each forward step multiplies the data by √(1−β_t) and adds Gaussian noise with variance β_t. Unrolled, x_t = √ᾱ_t·x_0 + √(1−ᾱ_t)·ε̂_t, and nothing here is learned. The reverse direction is learned with a Gaussian p_θ(x_{t−1}|x_t): apply last lecture's ELBO to the whole trajectory, split it with the KL chain rule, and the loss at each step is ‖μ̃_t(x_t,x_0) − μ_θ(x_t,t)‖²/(2β̃_t). The lecture stops at this expression; simplifying it into noise prediction is left for Lecture 12.
The first half collapses the diffusion ELBO into one expression: every time step is a squared error for guessing the noise that was added, which gives a five-step training procedure and a one-loop sampler. The second half opens the foundation-model unit: the two-stage paradigm of pretraining plus adaptation, linear probe, finetune, LP-FT, and why LoRA trains only 2dr parameters yet saves very little compute.
This lecture unpacks the SimCLR loss into a B×B similarity matrix: each row is one softmax classification problem whose answer is the diagonal entry. The diagonal gets pulled up and everything else gets pushed down, so the model learns that similar things have nearby vectors without any labels. The second half uses the same vectors for semantic retrieval, then connects that to RAG.
This lecture takes a large language model apart in three layers: tokenization turns text into token IDs, the chain rule splits the probability of a whole text into a sequence of next-token conditionals, and training sums the cross-entropy at every position. The second half opens up the Transformer: inner products of queries and keys go through a softmax to give weights, the weights average the values, and a causal mask blocks future positions. The cost of a single attention layer grows with the square of the sequence length T.
During generation, every head has to keep the keys and values of all past positions in GPU memory, and this KV cache caps the batch size. GQA lets τ query heads share one set of keys and values, shrinking the cache to 1/τ; sliding window attention cuts the T² compute down to T·w; MoE runs only 8 of 128 experts for each token. The second half covers how to use a trained model: few-shot and zero-shot prompting leave the parameters alone, SFT changes them, and its loss is computed only on the answer tokens.
The first 55 minutes of this lecture write sequential decision making as an MDP (states, actions, transition probabilities, rewards, discount factor) and show that the Bellman equation for a fixed policy is a system of linear equations. The last 20 minutes derive policy gradient: ∇p = p∇log p turns the gradient back into an expectation, the transition probabilities drop out of the gradient because they don't contain θ, and the lecture stops at the formula "sum of log-probability gradients × sum of discounted rewards", leaving variance reduction for the next lecture.
The last lecture of the term finishes policy gradient: past rewards and any baseline that depends only on the state can be removed from the weight without changing the expectation. It then derives PPO's clipped objective from importance sampling, splits it into four cases to see which samples still carry a gradient, and compares the change CISPO makes. Finally it writes token-by-token generation in a language model as an MDP, gives a reward of 1 only when the answer is right, and trains long chains of thought using the average over 8 samples of the same question as the baseline.