RNN Language Models: Sequence Probabilities and Gradient Paths | CMU 10-423 Lecture 1
A sequence such as “I like cats” is not a bag of independent words.
#n-gram
5 postsWord Embeddings and Language Models, from Counting N-grams to RNNs | NTHU Hung-Yu Kao Natural Language Processing Week 2
Week 2 of Hung-Yu Kao's NLP course at NTHU is a 62-page deck about predicting the next word. The first part covers statistical language models with a bigram count table, add-one smoothing, and perplexity, then sparse vectors through a PPMI example with cherry and digital. The middle returns to Word2Vec's negative sampling and uses the Chinese word for 'apple' to show why contextualized embeddings are needed. The last part derives RNNs from three weaknesses of feedforward networks and shows how they handle NER, sentence classification, and stacked and bidirectional variants.
Language — N-grams, Parser, Attention & the Transformer | Harvard CS50 AI Week 6
Week 6 is about getting computers to handle natural language: parse sentences with hand-written context-free grammar rules, then switch to statistics, from n-grams and Markov-chain text generation to a Naive Bayes classifier for review sentiment; turn words into vectors with word2vec, process whole sequences with an RNN encoder–decoder, and finish with attention and the Transformer. Includes section-by-section time links, plus an approach, common mistakes and ways to check your work for the Parser and Attention projects.
From Tokenization to N-gram Language Models | CMU 07-280 Lecture 17
Lecture 17 first decides how text becomes tokens, then uses N-grams to turn sequence probability into conditional probabilities estimated from corpus counts. Tokenization is the first design decision about what a model can see.
How N-grams Train, Sample, and Fail | CMU 07-280 Lecture 18
Lecture 18 truncates the chain rule with an N-gram Markov assumption, estimates probabilities from corpus counts, and contrasts greedy, categorical, and temperature sampling. The real bottlenecks are zero probability for unseen contexts and a fixed window.