MIT 6.7960 PS1 Walkthrough: From NumPy MLP to PyTorch Autograd Backprop
Hand-write NumPy MLP + backprop → verify with PyTorch Autograd, fully reproducing OCW HW1 core concepts
#backpropagation
7 postsHarvard CS50 AI Week 5: Neural Networks — Backpropagation, TensorFlow/Keras, CNN & Traffic Sign Classification
Week 5 enters deep learning: perceptron to multi-layer nets, backprop chain rule, loss functions, optimizers, TensorFlow/Keras modeling, CNN conv/pool. Project Traffic trains CNN to classify traffic signs.
How Models Improve Themselves: Gradient Descent and the Training Loop
A model uses loss to know how wrong it is and gradients to know which direction to adjust. Gradient descent repeats three things: compute loss, compute gradients, update parameters. The learning rate controls step size — too large and you overshoot, too small and training takes forever.
CMU 07-280 Lecture 12: How Backpropagation Reuses the Chain Rule
Lecture 12 treats a network as a computation graph: the forward pass stores intermediates, the backward pass propagates upstream gradients, and local linear, activation, and softmax rules compute every parameter gradient efficiently.
CS224N Lecture 3: Matrix Calculus and Backpropagation
Lecture 3 decomposes neural-network training into computation graphs, local derivatives, and the chain rule: the forward pass computes a result; backprop accumulates gradients from the output so every parameter knows how to move.
Deep Learning: Modules, Backpropagation, and Vectorization
Chapter 7 decomposes neural networks into composable modules and uses backpropagation and vectorization to explain how deep models can be trained efficiently.
CMU 10-301 HW5: Expose Neural Networks and Backpropagation with NumPy
HW5 avoids automatic differentiation so learners must track forward shapes, caches, and backward gradients themselves.