Models trained on large amounts of web data have three kinds of problems, each paired with a goal. Bias comes from the model attending to the majority and sacrificing minorities, and the goal is fairness; bias can enter at any step from data to evaluation, and can be addressed through data, the loss function, or interface design. Harmful content sits in the long tail of the data distribution, and the goal is safety; safeguards can be placed at four levels (training data, input, training, and output), and jailbreaking is the attempt to get around them. For hallucination the goal is factuality; a long answer can be split into minimal claims that are checked one by one, and that score can be fed back to train the model. The usual corrective for all three is alignment, yet a reward model is imperfect, optimizing against it too far makes things worse, and preference data carries biases of its own.
HW1 in ADL Fall 2025 gives a question and four Chinese paragraphs. The model first picks the relevant paragraph (paragraph selection, framed as four-way multiple choice), then marks the answer's start and end inside it (span selection), scored by Exact Match. The spec slides point you straight at Hugging Face's run_swag_no_trainer.py and run_qa_no_trainer.py. The simple baseline uses bert-base-chinese, length 512, effective batch size 2, and learning rate 3e-5, and both stages together take under three hours on an 8GB RTX 3070. The Kaggle leaderboard closed 9/29, and code plus report were due 10/1 on NTU COOL. Outside readers cannot get the Kaggle data or grading, but the task design, baseline settings, and five report questions are all usable for practice.
Lecture 11 of ADL Fall 2025 builds on the EMNLP 2024 Language Agents tutorial. It defines an agent as an entity that perceives and acts, then names what is new about language agents: reasoning itself counts as an internal action. The lecture is organized around three concepts. Reasoning covers CoT and ReAct; memory covers Generative Agents and its recency / importance / relevance retrieval; planning goes from greedy reactive planning to tree search and world models. It closes with multi-agent systems in three steps: initialization, orchestration, and team optimization.
The first ADL lecture has no math and sets up two ideas. First, machine learning gives a machine many input-output pairs and lets it find the function that turns one into the other; deep learning chains many simple functions into a production line, supplies data only at the two ends, and lets the machine learn what every station does. Second, to solve a problem with deep learning, write down the input X and the output Y: the form of the output decides whether the task is classification or sequence prediction, and the properties of the input decide which network fits. Along the way it answers two questions: why the breakthrough waited until after 2010 (data and GPUs), and why networks go deep instead of fat.
Training a model means answering three questions. What is the model: layers of neurons, each computing z = Wa + b followed by a non-linearity; one neuron draws only one boundary, so it cannot compute XOR, and stacking layers can. What does a good function mean: define a loss, and smaller is better. How to pick the best: gradient descent takes a step along the slope where it stands, and practice uses mini-batch SGD. With millions of parameters, gradients come from backpropagation: a forward pass stores every layer's output, a backward pass computes δ from the output layer back, and their product is each weight's gradient. The guide works each of these through with numbers.
Lecture 9 of ADL Fall 2025 answers two questions. The model gives you a probability distribution at every step, so how do you pick a word from it? And once you have a sentence, how do you judge it? The slides start with teacher forcing and exposure bias to show the gap between training and generation, then compare greedy, beam search, sampling, top-k, and nucleus sampling, and file temperature and the penalties under 'control' rather than decoding algorithms. The evaluation half covers BLEU, ROUGE, perplexity, and LLM-Eval, then explains why you would use RL to optimize whole-sentence quality directly.
A pre-trained model continues text well and still may not do what an instruction asks. The first half of this lecture covers the two steps of post-training: instruction tuning uses many different tasks to teach the model to read task descriptions, and RLHF then moves the model toward whole responses that people prefer, with a reward model and pairwise comparisons handling the cost of human labor and inconsistent scores, while DPO and KTO drop the reinforcement-learning part. InstructGPT and ChatGPT are these two steps chained together. The second half covers LLM adaptation: models are too large to fine-tune in full, so only a small number of parameters are trained, with Adapter, LoRA, and prompt tuning, and no single method is best on every task.
Pre-training lets a model learn general abilities from large unlabeled data before the task is known. There are three architectures: encoders see both directions and suit understanding; decoders see only the left, can generate, and are the type that won; encoder-decoders pre-train by denoising. Pre-training plus fine-tuning hits two obstacles, scarce labels and oversized models. GPT-3's in-context learning performs tasks with no parameter update, followed by hard prompts (a template plus a verbalizer, LM-BFF) and soft prompts that train vectors directly (P-Tuning, Prefix-Tuning). Soft prompts did better at the time, but what survived is prompting with no training at all.
A language model stores knowledge in its parameters: rare facts are poorly remembered, events after training are unknown, internal company documents were never seen, and answers cannot be checked. RAG lets it take the exam open-book: retrieve relevant passages for the question first, then hand them to the model with the question. This lecture covers how retrieval is done (chunking, sparse and dense retrieval, bi-encoders and cross-encoders, training a retriever without labeled pairs), what can be added before and after retrieval (query rewriting, reranking, multiple rounds of retrieval), the lines along which RAG developed (what to retrieve, how to use it, when to retrieve), and finally which situations suit prompting, RAG, or fine-tuning.
With pre-training data close to exhausted, models now improve by thinking more before they answer. Lecture 12 of NTU ADL covers this in two halves. The first is test-time scaling: the model stays fixed and inference spends more compute, through short CoT, parallel sampling followed by selection (majority voting, answer confidence, an outcome reward model), step-level scoring with a process reward model combined with search, and sequential revision. The second is learning to reason: the weights are updated, with reasoning data written by people, generated by the model and filtered, or produced by a stronger teacher model, or with reinforcement learning that only tells the model whether its answer was right, as in DeepSeek-R1-Zero.
This lecture has one through-line: a language model estimates the probability of a word sequence by predicting the next word one step at a time. It first covers how to represent a word, from dictionaries, one-hot vectors, and co-occurrence matrices to low-dimensional word embeddings. It then covers three generations of language models: n-grams count, and unseen combinations get zero probability; a neural LM predicts with a network, and similar words share probability automatically; an RNN LM accumulates all previous words in a hidden state and shares one set of weights across time steps. RNNs are trained with BPTT, multiplying by the same matrix many times makes gradients vanish or explode, clipping handles explosion, and the gating in LSTM and GRU handles vanishing. Finally, applications are divided by the shape of input and output into whole-sequence classification, tagging, and encoder-decoder.
The ADL Fall 2025 course page schedules seven TA recitations across ten videos: Colab and PyTorch, ipdb debugging → the life of an NLP project → data formats for four NLP task types → LoRA and QLoRA → scaling laws, the LLaMA architecture, and MoE → quantization and evaluation → deployment with vLLM. The videos were recorded by the TA in 2023 and 2024 and reused in Fall 2025, so the Homework 1 and Homework 3 they mention are those years' assignments. The course page's five slide links all return 404; files with the same names open under the Fall 2024 path, and Deployment has a video only. This guide follows what the TA actually demonstrates and warns about in each session, with examples you can work by hand.
With T hypotheses gₜ in hand, you can do more than pick one by validation: give each one vote (uniform blending), learn a set of ballot counts (linear blending), or learn another model that combines them (the form the handout calls any blending, also named stacking). This lecture proves in six lines of algebra that, under squared error, the averaged G is never worse than the average performance of the gₜ, and the gap is exactly how spread out the gₜ are, which is the variance. Finally, bootstrapping re-samples T data sets from the one data set you have, trains one gₜ on each, and lets them vote; that is bagging.
Foundations spends 16 lectures on four questions: when machines can learn, why they can (Hoeffding, growth functions, VC dimension), how (linear and logistic regression, nonlinear transforms), and how to learn better (regularization, validation). Techniques spends 16 more on three ways of handling features: kernels embed features in high dimensions (SVMs), aggregation combines hypotheses (AdaBoost, random forests, GBDT), and extraction learns hidden features (neural networks, matrix factorization). All 130 YouTube videos are free; practice with the Fall 2024 HW0–HW7.
A decision tree is a model that learns conditional aggregation as it goes: every path from the root to a leaf is a condition, and the leaf is a constant. C&RT splits the data in two with one decision stump at a time, picks the split with the smallest weighted impurity, and keeps going until nothing can be split. A tree grown this way almost always has E_in equal to 0, so it is pruned with "E_in plus λ times the number of leaves", with λ chosen by validation. The same recursion also handles multiclass problems, categorical features, and missing values.
This lecture first shows that the data D alone cannot determine the answer outside D (No Free Lunch), then adds one assumption: the data and the future inputs are drawn independently from the same distribution. Hoeffding's inequality then guarantees that, for a hypothesis h fixed in advance, the probability that E_in and E_out differ by more than ε is at most 2exp(−2ε²N). An algorithm has to choose among M hypotheses, so the bound is multiplied by M and becomes 2M exp(−2ε²N); with finitely many hypotheses, enough data, and a small E_in, learning is feasible.
Techniques Lecture 16 has no new model. It re-sorts the models of the first 15 lectures by three questions: where the features are hidden (kernel, aggregation, extraction, low-dimensional projection), how the error is brought down (gradients, equivalent problems, multiple steps), and how to hit the brakes when the model is too powerful (regularization, validation). This article works PCA plus AdaBoost-Stump on eight points from start to finish and checks the answer against each of the three questions; it closes by keeping the ReLU, Xavier/He initialization, momentum, RMSProp, and Adam material that the Fall 2024 on-campus course added.
Fall 2024 had six Foundations assignments. HW0 is 20 multiple-choice math prerequisite questions. HW1–HW5 each have 12 problems plus a bonus: Q1–4 are auto-graded, Q5–12 are graded by TAs, the programming problems use rcv1, cpusmall, and mnist from the LIBSVM datasets site, and HW5 uses LIBLINEAR. HW1 and HW2 each include a problem where you argue with a ChatGPT-style answer. Fall 2026 has released hw0 and hw1: hw1 is now 16 multiple-choice problems with 4 secretly chosen for TA grading, and the data is the course's own hw1_train.dat. The new policy allows AI tools and vibe coding, but AI-generated code needs block-by-block comments in your own words. Neither semester publishes official solutions.
The ξₙ of the soft-margin SVM can be written directly as max(1 − yₙ(wᵀzₙ + b), 0), which turns the whole problem into ½wᵀw plus C times the sum of hinge errors: an L2-regularized model in which C corresponds to 1/(2λ). The hinge error and logistic regression's cross-entropy are both convex upper bounds of the 0/1 error, so the SVM approximates L2-regularized logistic regression. There are two routes to probability outputs: Platt's two-level learning learns only two extra numbers, A and B; the representer theorem guarantees that the optimal w is a linear combination of the zₙ, so kernel logistic regression can be done directly in the Z space, at the price that the N coefficients βₙ are usually all nonzero.
The dual SVM's variable count no longer depends on the feature dimension d̃, but every entry of the Q matrix is still one d̃-dimensional inner product. This lecture merges "transform + inner product" into a single kernel function K(x, x′); for the second-order transform the inner product takes only O(d). Once training, computing b, and prediction all use K, the SVM never touches the Z space. Pushed to the limit, the same move gives the Gaussian kernel, which corresponds to an infinite-dimensional transform, at the price that too large a γ still overfits.