🌏 中文版
The official handout is titled Homework 7: Deep Learning and contains written plus programming work. Written sections cover CNNs, RNNs, Transformers and AutoDiff, and empirical questions. Programming implements an RNN cell, self-attention, hybrid language model, validation, and generation on a TinyStories subset. The ZIP supplies rnn.py, public tests/test data, a tokenizer, tiny train/validation stories, loss/metric references, a Colab notebook, and an environment file.
Establish a diagnostic order
Fix a tiny dataset and random seed. Check shapes, initial loss, one-step gradients, and tiny-set overfitting before architecture comparisons. If training loss does not move, inspect activations, normalization, and learning rate. Discuss regularization only when training succeeds and validation fails.
First executable action and completion
Start with the bundle's environment file:
conda env create -f environment.ymlThen run test_runner.py and validate/generate on tiny stories. Completion means passing public tests for the RNN cell, attention, hybrid LM, training, validation, and generation, and reproducing tiny loss/metric references within their expected tolerance. Public tests do not replace Gradescope hidden tests.
Practice diagnosis from loss records
This is an invented teaching example, not an official assignment solution.
Two invented runs: A’s training loss falls from 1.0 to 0.2 while validation rises from 1.0 to 1.3; B’s two losses stay at 1.0. A fits training better while validation worsens: inspect overfitting and distribution issues. B has not fit training: inspect the training path and gradients first.
Check and answer: Does B alone prove insufficient network capacity? No. First check weight updates, nonzero gradients, and correct data/labels; then test fitting a tiny subset to narrow the diagnosis.
Changelog
- 2026-10-11: Added an invented example, worked calculation or reasoning, and a comprehension check with an answer.
References
Loading...