Skip to content
All tags

#cmu

73 posts
ai guide Reading CMU 07-280

Completing CMU 07-280: What You Know, What Is Missing, and What Comes Next

Finishing 07-280 means more than reading 24 guides: produce a search engine, supervised-model comparison, CNN/GPT-2 experiments, and a small RL-plus-MCTS system before choosing 07-380, 10-301, or a specialist course.

Reading CMU 07-280: Why Search, GPT-2, and AlphaZero Belong in One Course

07-280 is CMU's new Spring 2026 AI+ML core: 24 lectures and 12 main assignments move from heuristic search and CSPs to AlexNet, GPT-2, and AlphaZero. Its public material supports self-study, but complete recordings, Canvas checkpoints, Gradescope, and staff feedback remain unavailable.

CMU 07-280 Lecture 1: The Shared Problem Behind AI, ML, and Representation Learning

Lecture 1 uses an alien autoencoder, the scope of AI and ML, and AI history to establish the course's coordinate system: an intelligent system turns inputs into representations and decisions under uncertainty.

CMU 07-280 Lecture 2: Heuristic Search from UCS and Greedy to A*

Lecture 2 decomposes search into a problem, frontier, and priority: UCS uses paid cost, Greedy uses estimated remaining cost, and A* combines them as `f=g+h`; tree and graph search require different optimality conditions.

CMU 07-280 Lecture 3: Minimax, Alpha-Beta, and Expectimax

Lecture 3 turns a single path into a contingent plan: minimax faces an optimal opponent, alpha-beta skips branches without changing the root value, and expectimax replaces worst-case choice with probability.

CMU 07-280 Lecture 4: CSPs, AC-3, and Search Order

Lecture 4 exposes structure through variables, domains, and constraints, then upgrades DFS with backtracking, forward checking, AC-3, MRV, and LCV; the goal is to prove failure earlier.

CMU 07-280 Lecture 5: Defining Machine Learning with Loss, Risk, and ERM

Lecture 5 formulates machine learning through `X → Y`, loss, risk, and empirical risk minimization: a training set only gives average observed loss, while the real objective remains generalization over an unknown distribution.

CMU 07-280 Lecture 6: How Decision Trees Split Data with Mutual Information

Lecture 6 recursively grows a tree from decision stumps, measures label uncertainty with entropy, and selects splits by `I(Y;W)=H(Y)-H(Y|W)`; this is computationally practical greedy ERM, not a global optimal-tree guarantee.

CMU 07-280 Lecture 7: Linear Regression and the Normal Equation

Lecture 7 applies ERM to linear functions and squared loss, moves from a one-dimensional slope to `argmin ||y-Xθ||²`, and derives the normal equation when `XᵀX` is invertible.

CMU 07-280 Lecture 8: Gradient Descent, SGD, and Learning Rate

Lecture 8 moves from a one-dimensional parabola to vector gradients and compares batch GD, SGD, and mini-batches; the learning rate determines whether updates converge, oscillate, or diverge.

CMU 07-280 Lecture 9: Logistic Regression as Probability Estimation

Lecture 9 models P(y=1|x) with a sigmoid instead of directly predicting 0 or 1, learns parameters with cross-entropy and convex optimization, and extends naturally to softmax regression.

CMU 07-280 Lecture 10: Trading Expressiveness for Stability with Features and Regularization

Lecture 10 uses φ(x) to let linear models express nonlinear functions, then controls the resulting overfitting with train/validation/test separation, L1/L2 regularization, and model selection.

CMU 07-280 Lecture 11: Building a Neural Network from Logistic Regression

Lecture 11 expands a logistic unit into a multilayer network: linear layers produce z, activations produce a, and multiple neurons jointly learn a feature transform trained through a final loss.

CMU 07-280 Lecture 12: How Backpropagation Reuses the Chain Rule

Lecture 12 treats a network as a computation graph: the forward pass stores intermediates, the backward pass propagates upstream gradients, and local linear, activation, and softmax rules compute every parameter gradient efficiently.

CMU 07-280 Lecture 13: From Reward Hacking to Auditable AI Scientists

Lecture 13 separates alignment into specification, distribution shift, oversight, and corrigibility, then uses benchmark selection, leakage, and post-hoc selection experiments to show why a final paper cannot audit an autonomous research workflow.

CMU 07-280 Lecture 14: Encoding Image Structure with Convolutional Networks

Lecture 14 replaces dense image models with local connectivity and parameter sharing, moving from convolution, stride, padding, and pooling to AlexNet, GPU data parallelism, ResNet skip connections, and BatchNorm.

CMU 07-280 Lecture 15: Separating Pretraining, Transfer Learning, and Fine-Tuning

Lecture 15 splits a pretrained model into representation g and task head h: freeze g and train only the head, or fine-tune some or all parameters at a smaller learning rate depending on data volume and source-target distance.

CMU 07-280 Lecture 16: Unifying Logistic and Linear Regression with Maximum Likelihood

Lecture 16 starts from likelihood p(D|θ), uses i.i.d. to factor the joint probability and logs to turn products into sums; Bernoulli MLE yields sample proportions, conditional Bernoulli yields logistic cross-entropy, and Gaussian noise yields squared error.

CMU 07-280 Lecture 17: From Tokenization to N-gram Language Models

Lecture 17 first decides how text becomes tokens, then uses N-grams to turn sequence probability into conditional probabilities estimated from corpus counts. Tokenization is the first design decision about what a model can see.

CMU 07-280 Lecture 18: How N-grams Train, Sample, and Fail

Lecture 18 truncates the chain rule with an N-gram Markov assumption, estimates probabilities from corpus counts, and contrasts greedy, categorical, and temperature sampling. The real bottlenecks are zero probability for unseen contexts and a fixed window.

CMU 07-280 Lecture 19: Turning Next-token Prediction into Geometry

Lecture 19 builds a minimal next-token model from two embedding matrices, dot-product similarity, softmax, and cross-entropy. Shared vector parameters replace the isolated count cells of an N-gram table.

CMU 07-280 Lecture 20: From Position Encoding to Causal Self-Attention

Lecture 20 expands one-token embeddings into sequences, adds positional information, derives Q/K/V scaled dot-product attention and causal masking, and assembles multi-head blocks into a GPT-2 skeleton.

CMU 07-280 Lecture 21: How Bellman Equations Solve Markov Decision Processes

Lecture 21 formulates stochastic sequential decisions as an MDP with known dynamics, defines value and Q-values through Bellman backups, and solves for an optimal policy with value or policy iteration.

CMU 07-280 Lecture 22: Q-learning When Dynamics Are Unknown

Lecture 22 keeps the MDP structure but removes known transitions and rewards. TD learning updates value from one sample, and Q-learning uses an off-policy target to learn optimal action values directly.

CMU 07-280 Lecture 23: From Approximate Q-learning to DQN

Lecture 23 replaces a huge Q-table with Qθ(s,a): first derive a gradient update for linear features from squared TD error, then add replay data and a fixed target network to form DQN.

CMU 07-280 Lecture 24: How Monte Carlo Tree Search Connects to AlphaZero

Spring 2026 Lecture 24 is MCTS, not Fall 2026 LLM post-training. It allocates simulations through selection, expansion, rollout, backup, and UCB, then connects policy/value heads and self-play to AlphaZero.

CMU 07-280 Stage Review I: From Search Problems to Supervised Learning

Lectures 1–12 form one decision pipeline: define states, moves, and objectives, then use heuristics, losses, regularization, and backpropagation to control an otherwise intractable search space.

CMU 07-280 Stage Review II: Building AlexNet and GPT-2 as Working Systems

Stage II uses HW8 and HW11 to test whether representation, computation graphs, training, transfer, and generation actually connect, rather than treating CNNs and Transformers as diagrams to memorize.

CMU 07-280 Stage Review III: From MDPs and Q-learning to AlphaZero

Stage III connects value, policy, bootstrapping, function approximation, and MCTS into AlphaZero: a network supplies priors and estimates, search improves decisions, and self-play creates the next training set.

CMU 11-785 Lecture 1: Introduction

Spring 2026 Lecture 1 focuses on neurons, perceptrons, connectionism, and the problem framing of deep learning. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 2: Neural Nets as Universal Approximators

Spring 2026 Lecture 22 focuses on latent variables, the ELBO, the KL term, and the reparameterization trick. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 3: Training I: Learning and Empirical Risk Minimization

Spring 2026 Lecture 3 focuses on data distributions, hypotheses, losses, empirical risk, and their roles in generalization. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 4: Training II: Gradient Descent

Spring 2026 Lecture 4 focuses on gradients, learning rates, parameter updates, and the training of a linear neuron. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 5: Training III: Backpropagation

Spring 2026 Lecture 5 focuses on computational graphs, the chain rule, local derivatives, and gradient reuse. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 6: Training IV: Convergence, Loss Surfaces, and Momentum

Spring 2026 Lecture 6 focuses on non-convex loss surfaces, curvature, saddle points, and momentum's accumulated direction. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 7: Training V: SGD and Second-order Methods

Spring 2026 Lecture 7 focuses on the tradeoffs among full-batch, mini-batch, stochastic gradients, and second-order information. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 8: Training VI: Optimizers and Regularization

Spring 2026 Lecture 8 focuses on AdaGrad, Adam, regularization, BatchNorm, Dropout, and loss selection. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 9: CNNs I

Spring 2026 Lecture 9 focuses on local connectivity, weight sharing, convolution kernels, and feature maps. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 10: CNNs II

Spring 2026 Lecture 10 focuses on stride, padding, receptive fields, and multi-channel convolution. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 11: CNNs III

Spring 2026 Lecture 11 focuses on stacked convolutional architectures, feature hierarchies, and design tradeoffs. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 12: CNNs IV

Spring 2026 Lecture 12 focuses on CNN training, architecture selection, and the end-to-end assembly of a vision model. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 13: RNNs I

Spring 2026 Lecture 13 focuses on sequence state, temporal unrolling, parameter sharing, and recurrent computation. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 14: RNNs II

Spring 2026 Lecture 14 focuses on backpropagation through time, gradient stability, and LSTM-style gated memory. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 15: Seq2Seq and Connectionist Temporal Classification

Spring 2026 Lecture 15 focuses on variable-length input/output, unknown alignment, and the CTC objective. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 16: CTC Blanks and Beam Search

Spring 2026 Lecture 16 focuses on blanks, collapse rules, prefix probabilities, and approximate decoding. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 17: Language Models and Translation

Spring 2026 Lecture 17 focuses on autoregressive factorization, conditional language models, and translation decoding. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 18: Attention and Transformers

Spring 2026 Lecture 18 focuses on queries, keys, values, scaled dot-product attention, and the Transformer block. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 19: Transformers and Newer Architectures

Spring 2026 Lecture 19 focuses on encoder/decoder structures, masks, residual paths, and architecture variants. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 20: Large Language Models

Spring 2026 Lecture 20 focuses on scaled autoregressive models, training stages, inference, and capability boundaries. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 21: Representations and Autoencoders

Spring 2026 Lecture 21 focuses on bottleneck representations, reconstruction objectives, dimensionality reduction, and representation quality. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 22: Variational Autoencoders

Spring 2026 Lecture 22 focuses on latent variables, the ELBO, the KL term, and the reparameterization trick. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 23: Diffusion Models

Spring 2026 Lecture 23 focuses on forward noising, reverse denoising, score or noise prediction, and sampling. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 24: Generative Adversarial Networks

Spring 2026 Lecture 24 focuses on the generator, discriminator, minimax objective, and training instability. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 25: Graph Neural Networks

Spring 2026 Lecture 25 focuses on message passing, aggregation, node representations, and permutation symmetry. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 26: Reinforcement Learning

Spring 2026 Lecture 26 focuses on states, actions, rewards, returns, values, and policy learning. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 27: Hopfield Networks

Spring 2026 Lecture 27 focuses on associative memory, energy functions, fixed points, and pattern retrieval. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

CMU 11-785 Lecture 28: Boltzmann Machines

Spring 2026 Lecture 28 focuses on energy-based probability models, stochastic units, the partition function, and learning difficulty. This guide follows the official slides and recording and adds a small self-check that does not depend on the enrolled-course grader.

A Complete Guide to CMU 11-785: 28 Public Lectures, but an Incomplete Assignment Chain

CMU 11-785 Spring 2026 publishes official slides and YouTube recordings for all 28 content lectures, plus extensive bootcamps and recitations. Its HW1–HW4 specifications, starters, and evaluation still depend on Autolab, Piazza, and Kaggle.

CMU 07-380 Fall 2026 Overview: 26 Lectures from Logic and Planning to Diffusion, HW and Project Not Yet Fully Released

07-380 Fall 2026 is the first offering of CMU's new AI II, 26 lectures from logic, planning and optimization to probabilistic graphs and generative systems; Lec01 and Prop Logic are public, HW1-7, six quizzes and the final project are still TBD — an A2→A3 transition with the 07-280 bridge.

CMU 10-301 HW1: Find ML Foundation Gaps with Mathematics and Python

HW1 is written and programming work: mathematical and CS foundations followed by a majority-vote classifier.

CMU 10-301 HW2: From Information Calculations to a Complete Decision Tree

HW2 moves from hand-calculated entropy and mutual information to an end-to-end tree learner, predictor, and evaluator.

CMU 10-301 HW3: Compare K-NN, Perceptron, and Linear Regression

HW3 is written work: a decision-tree review followed by K-NN, Perceptron, and Linear Regression through inductive bias, errors, and model selection.

CMU 10-301 HW4: Turn Logistic Regression Likelihood into a Classifier

HW4 joins probabilistic interpretation, cross-entropy gradients, and implementation into one traceable training pipeline.

CMU 10-301 HW5: Expose Neural Networks and Backpropagation with NumPy

HW5 avoids automatic differentiation so learners must track forward shapes, caches, and backward gradients themselves.

CMU 10-301 HW6: Learning Theory, MLE/MAP, and Fairness Metrics

HW6 combines generalization, MLE/MAP, probabilistic learning, fairness metrics, and social impact in one written assignment about assumptions and tradeoffs.

CMU 10-301 HW7: Move from Basic Neural Networks to Deep Learning

HW7 builds on HW5 backpropagation to address deep-model architecture and training failures, emphasizing diagnosis over merely adding layers.

CMU 10-301 HW8: From MDPs to Reinforcement-Learning Updates

HW8 connects states, actions, rewards, transitions, and value updates while separating environment dynamics, policy, and estimation error.

CMU 10-301 HW9: Close the Course with Ensembles, k-Means, PCA, and Recommenders

The final written assignment combines ensembles, clustering, representation, and recommendation to test whether you can choose a learning paradigm from problem structure.

CMU 10-301/601 Spring 2026: Learn Machine Learning Through Nine Assignments

Spring 2026 publishes material for 27 lectures and nine homework bundles; outsiders can do the core work but cannot access Panopto, Piazza, Gradescope, or official homework solutions.

learning deep-dive

CMU's AI Core Redesign: From 15-281 + 10-315 to 07-280 + 07-380

In 2026, CMU recombined its separate general-AI and SCS machine-learning introductions into the 07-280 → 07-380 sequence. This is a redistribution of content and prerequisites, not a pair of simple course renames.

learning deep-dive

The Pacman AI Project Lineage: How Berkeley CS188 and CMU 15-281 Restructure the Same Material

CMU 15-281's Search and Games explicitly credits Berkeley's Pacman AI projects. The official course site separately lists a zero-point P0 tutorial and five programming assignments, P1–P5.

learning deep-dive

CMU's AI Degrees: The First U.S. AI Bachelor's Turned 'What Should AI Students Learn?' into Graduation Requirements

Stanford has no AI degree; AI is a track inside CS. CMU launched the first U.S. B.S. in Artificial Intelligence in 2018, divided AI into four clusters, required one course from each, and made ethics a graduation requirement. At the master's level, MSAII sits not in CS but in the Language Technologies Institute; 84 of its 195 units cover an innovation process ending in a fundable capstone. Two official-page conflicts emerged during verification: whether the AI Core has two or three courses, and whether MSAII totals 192 or 195 units.

CMU AI/ML Course Guide: The New 07-280 Core and a Public Self-Study Route

CMU's current BSAI now runs through 07-280 and 07-380 before branching into an NLP/vision core and four AI clusters, but 07-380 does not debut until Fall 2026. The residual Spring 2026 materials for 07-280 and the complete 10-301/601 site already support self-study; retired 15-281 remains a useful legacy route.