Memory and Sequence Modeling — RNNs, LSTMs, and Vanishing/Exploding Gradients | MIT 6.7960 Lecture 10
RNNs propagate hidden states through time; gradient products can shrink or grow. LSTM gates control cell-state retention and updates, improving long-range signal flow without guaranteeing stability. Self-attention shortens paths between positions, while dense attention still has quadratic sequence-length cost.