Skip to content
All tags

#rnn

2 posts
tech MIT 6.7960 導讀 (Fall 2024 OCW)

MIT 6.7960 L10: Memory and Sequence Modeling — RNNs, LSTMs, and Vanishing/Exploding Gradients

An RNN compresses the past into a hidden state, but recurrence makes gradients multiply over time — they either vanish or explode; LSTM decouples 'memory' from 'update' via input/forget/output gates so long-range information flows stably. Attention later replaced it because it reaches any history in O(1).

#mit-67960#deep-learning#rnn#lstm#sequence-modeling#fall-2024
ai deep-dive Reading Stanford CS224N

CS224N Lecture 4: Language Models, RNNs, and Vanishing Gradients

Lecture 4 defines a language model as a next-word probability distribution, then uses an RNN to compress an arbitrarily long prefix. It also exposes recurrence's central cost: information and gradients travel one time step at a time.

#cs224n#language-model#rnn#nlp#stanford