Initialization, BatchNorm, CNNs, Early Stopping, and Double Descent | Berkeley CS189 Lectures 19–20
Lec 19 wraps up backprop, then tackles how to keep gradients flowing: all-zero initialization makes every unit learn the same thing, so use small random values (He init for ReLU), and batch norm normalizes pre-activations with mini-batch means and variances. The second half introduces CNNs: local connectivity plus weight sharing lets one feature detector scan the whole image. Lec 20 finishes pooling, receptive fields, and CNN training, then covers early stopping, dropout, and double descent, which breaks the classic bias-variance picture.