🌏 中文版
Video status: Related supplementary video included; the original lecture recording has not been verified. Source details
This is article 23 in Reading Stanford CS109, covering Summer 2026 Lecture 22: Deep Learning (Jul 30), taught by Chris Gregg. It follows the official worksheet, answer key, and LLM guide.
This remains L2. The four-page worksheet has P1–P7 plus an optional multi-class challenge; the five-page key answers every problem with no pset omission. The three-page guide has six concepts. Current slides are unavailable and video is Canvas-gated.
P1: Calibration and baselines
Among 60 predictions near 0.8, only 36 are positive, an observed fraction of 0.60. The bucket is uncalibrated and overconfident. Since 70% of the dataset is label 1, the always-positive baseline scores 0.70. Calibration constrains what a probability means, not whether thresholded decisions beat a baseline.
P2: Softmax
Softmax is softmax(z)ₖ=e^zₖ/Σⱼe^zⱼ. It maps [7,7,7,7] to [0.25,0.25,0.25,0.25], and [1,3,0] to approximately [0.114,0.844,0.042]. Positive exponentials make every output positive and the common denominator makes them sum to one; the largest score gets the largest probability and no finite score receives exact zero.
For two classes, dividing by e^z₁ gives e^z₁/(e^z₁+e^z₂)=1/(1+e^{-(z₁-z₂)})=σ(z₁-z₂). Sigmoid is to Bernoulli as softmax is to Categorical. Renormalized sigmoids also yield probabilities, but lose the log-likelihood concavity used by the course.
P3: Counting parameters
With 40 inputs, 20 hidden units, and one output, excluding bias, the first layer has 40×20=800 weights and the second has 20, totaling 820. Adding another 20-unit layer gives 800+400+20=1220. Biases for the original network add 21 parameters, totaling 841; each is the logistic-regression intercept weight on a constant input.
P4: A forward pass
For bias-prepended x=[1,1,0], the hidden weighted sums are 2 and 0, hence h₁=0.8808 and h₂=0.5. The output sees h=[1,0.8808,0.5], giving z⁽²⁾=1.2616 and ŷ=0.7793. With y=1, LL=ln(0.7793)≈-0.2494.
The pass evaluates three logistic regressions: two hidden neurons using x, and one output neuron using hidden activations. “Learning features” means outputs from one layer become inputs to the next.
P5: Output-layer backpropagation
The Bernoulli likelihood is unchanged from Lecture 20. Chain-rule sigmoid factors cancel:
∂LL/∂θⱼ⁽²⁾ = hⱼ(y-ŷ)Here y-ŷ=0.2207, so bias-inclusive gradients are [0.2207,0.1944,0.1103]. An η=1 update raises ŷ to about 0.8467 and LL from -0.2494 to about -0.1664. The output layer only sees h, making this the logistic gradient with xⱼ replaced by hⱼ.
P6: Hidden-layer backpropagation
A buried weight affects zⱼ⁽¹⁾→hⱼ→z⁽²⁾→ŷ→LL. Multiplying four local derivatives yields:
∂LL/∂θⱼ,ₖ⁽¹⁾ = (y-ŷ)θⱼ⁽²⁾hⱼ(1-hⱼ)xₖFor x₁=1, the two requested gradients are approximately 0.0463 and 0.0552. Although h₂ has a smaller outgoing weight, h₂=0.5 lies at sigmoid's most sensitive point; h₁ is partly saturated. When x₂=0, its corresponding gradient is zero.
Backpropagation computes error at the output and sends it leftward through local derivatives. Reusing that signal avoids restarting differentiation for every weight. Tiny derivatives near saturated activations also reveal the source of vanishing gradients.
P7: Deep learning is MLE
The lecture's conclusion is: Deep learning is maximum likelihood estimation with neural networks. The steps remain: state a probability model, write all-data log-likelihood, then differentiate with respect to every parameter and optimize by gradient ascent.
A binary network and logistic regression both assume Y|X=x~Bern(ŷ); only the function computing ŷ changes from one layer to a composition. Neural-network objectives are non-concave, with local optima and saddle points, so initialization matters. Removing hidden nonlinearities collapses the composition of linear maps into one linear map and reduces the network to an elaborate logistic regression.
Optional challenge: Multi-class log-likelihood
For softmax outputs and a one-hot label:
LL = Σₖyₖlog ŷₖ = log ŷ_cOnly the correct class remains. At K=2 this is exactly Bernoulli log-likelihood. For the correct score, ∂LL/∂z_c=(1/ŷ_c)ŷ_c(1-ŷ_c)=y_c-ŷ_c; other classes also yield yₖ-ŷₖ. The recurring error-times-input form across Bernoulli, Categorical, and deep networks is structural rather than accidental.
Guide unit: Six concepts
The guide covers softmax, stacked logistic units, deep-learning likelihood, output backpropagation, hidden backpropagation, and non-concave optimization. Compute a forward pass first, then write every partial derivative along the dependency chain. Stay at the CS109 MLE and chain-rule level rather than jumping to frameworks or transformers.
Finally compare logistic regression and networks by expressive power, parameter count, concavity, interpretability, and data demand. More parameters are not automatically better; deployment still requires Lecture 21's held-out evaluation, calibration, and fairness audit.
Material boundaries
- The worksheet has P1–P7 plus a challenge; the five-page key answers all of them.
- The three-page guide has six concepts and no extra numbered unit.
- Current slides are unavailable and video is Canvas-gated; only public artifacts are used.
Course video sources
Verified the official Summer 2026 archive: the course is recorded and its syllabus says recordings are available through Canvas. The archived homepage's Videos link redirects to Stanford Canvas login and requires course access, so there is no public Summer 2026 player. The 2022 Stanford Online public recording(s) below cover the same topic but come from a different offering with different lecture numbering; they are supplementary material, not Summer 2026 lecture recordings.
Original videos: Stanford CS109 I Deep Learning I 2022 I Lecture 25
Official sources:
- CS109 Summer 2026 — Canvas lecture recordings (login required)
- CS109 Summer 2026 — archived official schedule
- CS109 Summer 2026 — archived syllabus and recording policy
- Stanford Online CS109 2022 public playlist (different offering, supplementary)
Checked: 2026-10-10.
Content check: verified against the video transcript (2026-10-10): Read the transcript of 2022 Lecture 25, "Deep Learning" (about 80K characters): the video presents a neural network as stacked logistic regressions trained with the sigmoid, MLE and the chain rule, matching this lecture's topic. Softmax is only mentioned briefly at the end (switching to softmax and a multinomial loss), and the word backpropagation never appears in the transcript, so the P2 softmax section and the layer-by-layer backprop hand calculations in P5 and P6 are not video content; there is also no calibration problem.
Update Log
- 2026-10-10: Added explicit video status and checked recording sources and access notes.
- 2026-10-10: Rechecked video status. Switched to related supplementary video: embedded the 2022 Stanford Online public recording on the same topic; the Summer 2026 original is behind Canvas sign-in and could not be verified.
- 2026-10-10: Checked the video content against its transcript. The video topic matches; softmax is only mentioned briefly, and backpropagation and the worksheet problems are not video content, as the check note says.
References
Loading...