A week-by-week reading of Stanford CS124: language models, text classification, information extraction, question answering, speech, and the full NLP pipeline.
CS124 is the first course in Stanford's NLP track: ten weeks from tokenization, n-grams, text classification, information retrieval and embeddings to LLMs and Transformers. The textbook is the free Speech and Language Processing, and the nine assignment repos plus the lab problems and solutions are public; the pre-recorded videos, quizzes and autograding need a Stanford account.
Week 1 has no algorithms. The live introduction explains what the course teaches: which week covers each component of an LLM, which language algorithms outside LLMs are also taught, and why language is hard. PA0 sets up the Python and Jupyter environment the later assignments use.
Week 2 teaches three things: cutting text into tokens with BPE, computing the edit distance between two strings with dynamic programming, and estimating the next word's probability with n-grams, evaluated by perplexity. Each can be done on paper, and this post works one example of each.
Week 3 teaches logistic regression: turn a document into a feature vector, pass the weighted sum through a sigmoid to get a probability, measure the error with cross-entropy, and fix the weights with gradient descent. This post works the slides' movie-review example through a prediction, a loss and one weight update, and explains why evaluation looks at precision and recall.
Week 4 teaches search: turn the query and the documents into vectors, weight them with tf-idf, rank by cosine similarity, speed it up with an inverted index, evaluate with precision, recall and MAP, and finally connect to dense retrieval and RAG. This post works the slides' four-document example in full.
Week 5 teaches embeddings: the meaning of a word can be represented by the words that tend to appear around it. It builds vectors from co-occurrence counts and compares them by cosine, then trains dense vectors of a few hundred dimensions with word2vec, and finally separates static vectors from ones that change with context. The same week's live lecture on computational social science has no public material.
Week 6 has two halves. The pre-recorded material teaches neural networks: how one unit computes, why hidden layers are needed, the softmax, classification and language modeling with word vectors, and backpropagation. The live lecture introduces large language models: what they are, the three architectures, why so many tasks can be recast as continuing text, how to sample, how pretraining works, and how a model becomes an assistant.
Week 7 opens up the Transformer: how attention lets each token draw on the tokens before it, what parts make up a block, and how the input and output ends are attached. PA6a has you write causal self-attention, train a small model and compute perplexity. The same week's live speech lecture introduces the speech recognition task and word error rate.
Week 8 continues with speech: how sound is produced, how a waveform becomes the log mel spectrum a model consumes, and how Whisper turns audio into text with an encoder-decoder and cross-attention. PA6b converts text to speech and back and analyzes what was lost. Lab 4 introduces the PA7 agent loop and teaches teams to collaborate with GitHub.
Week 9 teaches recommender systems: content-based recommendation looks at the features of items, while collaborative filtering looks only at the ratings matrix and estimates unrated items from similar users or similar items. PA7 writes item-item collaborative filtering as a function and hands it to an LLM agent that makes tool calls, then adds ticket booking, web search and memory.
Week 10 treats web pages and human relationships as graphs. Link analysis teaches anchor text and PageRank: the long-run probability that someone clicking links at random is on a page is that page's score. Social networks covers centrality, the clustering coefficient, small worlds and weak ties. The final live lecture returns to LLMs with preference alignment, interpretability and machine translation.