CMU 10-301 HW5: Expose Neural Networks and Backpropagation with NumPy

TL;DR HW5 avoids automatic differentiation so learners must track forward shapes, caches, and backward gradients themselves.

#cmu #machine-learning #neural-network #backpropagation

Table of Contents

Read code as a computation graph
First executable action and completion
References

Series: Reading CMU 10-301 Machine Learning (5 / 10)

CMU 10-301 HW4: Turn Logistic Regression Likelihood into a Classifier CMU 10-301 HW6: Learning Theory, MLE/MAP, and Fairness Metrics

🌏 中文版

The official handout is titled Homework 5: Neural Networks and contains written plus programming work. Written work uses a feed-forward/backpropagation calculation and empirical questions. Programming builds a one-hidden-layer sigmoid/softmax OCR letter classifier. The ZIP supplies neuralnet.py, small train/validation CSV files, tests.py, unit-test data, an incorrect-output finder, and a visualizer; medium/large data and the official reference solution are absent.

Read code as a computation graph

Label every tensor shape and cache only values required backward. Write local derivatives first, then accumulate along the graph. Batch dimensions, bias broadcasting, and loss normalization are common silent failures.

First executable action and completion

From the bundle's handout directory, run:

python -m unittest tests

Then use the handout's neuralnet.py interface on the small CSV files, run finite-difference checks, and overfit a tiny subset. Completion means passing public tests, producing correctly formatted per-epoch loss/labels/metrics, and matching numerical gradients. The handout explicitly says public tests are non-exhaustive, so this is not equivalent to full Gradescope credit.

CMU 10-301 HW5: Expose Neural Networks and Backpropagation with NumPy

Read code as a computation graph

First executable action and completion

References

Related