Lecture 2 breaks supervised learning into three pieces: a hypothesis h, the least-squares cost J(θ), and an algorithm that drives J down. Gradient descent sweeps all n examples on every step; minibatch SGD updates after looking at a small batch, and pays for it by bouncing around near the optimum. Linear regression also has a closed-form solution, θ = (XᵀX)⁻¹Xᵀy, provided XᵀX is invertible.
Linear regression writes the squared error as (1/N)‖Xw − y‖², sets the gradient to zero, and gets w_LIN = X†y with no iteration. The hat matrix H = XX† projects y onto the space spanned by the columns of X, and trace(I − H) = N − (d+1), which gives average E_in = σ²(1 − (d+1)/N) and average E_out = σ²(1 + (d+1)/N). Squared error is also an upper bound on the 0/1 error, so w_LIN can be used for binary classification too.
HW2 is a written-only assignment with 10 problems. The first half practices reading a paper (Chatbot Arena) and the core derivations for regression and MLE/MAP. The two heaviest problems come last: Mixed Feelings goes from k-means' fragility to outliers through robust k-means to a GMM with a uniform background, and Watch Me Flow Dat proves that conditional flow matching and a discretized MLE are the same objective. The problem PDF and LaTeX template are freely downloadable; there are no official solutions.
Over four lectures, CS189 Spring 2026 presents linear regression from three angles that meet in one formula: MLE under Gaussian noise is least squares; the least-squares solution is the orthogonal projection of y onto the column space of X; and when features are collinear or too many, ridge (the MAP estimate under a Gaussian prior) or lasso (a Laplace prior) pulls the solution back, with λ chosen on a validation set. Slides, videos, and Discussion 3–4 solutions are all publicly accessible.
HW0 checks CS181 prerequisites in four problems — y=Xw solvability, optimizing an objective, reasoning about randomness, and OLS in Python. The problem that slows you down most is the gap to patch before HW1.
HW1 uses an 800,000-year ice-core temperature dataset across four problems: kNN and kernel regression, a geometric proof of least squares, basis-function regression, and a probabilistic derivation of ridge and LASSO, ending with a coordinate-descent LASSO implementation.
Lecture 7 applies ERM to linear functions and squared loss, moves from a one-dimensional slope to `argmin ||y-Xθ||²`, and derives the normal equation when `XᵀX` is invertible.