Anthropic engineering lead Felix Rieseberg splits 'being more ambitious' into Speed, Altitude, Depth, and Range, then uses one quarterly business review (QBR) to show how to prompt and verify for each of the last three.
pstack is an MIT-licensed set of agent skills from Cursor engineer Lauren Tan. One entry point, /poteto-mode, picks one of 23 playbooks per task and calls other skills and 24 principles as steps need them. Install takes one command and one model setup; the price is extra tokens for subagents and review panels.
This lecture puts the first concrete content into the flowchart from Lecture 1: the hypothesis set H is all lines (perceptrons), and the algorithm A is PLA, which adds yₙxₙ to w whenever it sees a misclassified point. When the data is linearly separable, PLA started from w₀ = 0 halts after at most R²/ρ² corrections; when the data cannot be separated, switch to pocket, which keeps the line with the fewest mistakes seen so far in its pocket.
This lecture takes the learning flow diagram from Lecture 1 and swaps one part at a time along four directions: the output space Y (binary classification, multiclass classification, regression, structured learning), the labels yₙ (supervised, unsupervised, semi-supervised, reinforcement learning), the way data reaches the algorithm (batch, online, active), and the input space X (concrete, raw, abstract features). Each direction has one core option, and together they mark the home ground for the rest of Foundations: batch, supervised binary classification and regression on concrete features.
Once a hypothesis set has a break point k, the largest number of dichotomies it can produce on N points, B(N, k), satisfies B(N, k) ≤ B(N−1, k) + B(N−1, k−1). That gives B(N, k) ≤ C(N, 0) + … + C(N, k−1), whose highest term is N to the power k−1. Three more steps put the growth function into Hoeffding and yield the VC bound: the probability of the bad event is at most 4·m_H(2N)·exp(−ε²N/8). The break point of 2D perceptrons is 4, so they can learn.
This lecture replaces the fixed target function with a target distribution P(y|x): as long as each (x, y) is drawn independently from the same joint distribution, the VC bound holds with noise just as before. The error measure err and P(y|x) together decide the ideal prediction on each x; the 0/1 error picks the most probable y and the squared error picks the weighted mean. CIA fingerprint verification penalizes a false accept 1000 times more, and this kind of weighted classification is the same as copying the −1 examples 1000 times and then doing ordinary classification.
Logistic regression uses θ(wᵀx) to estimate P(+1|x), yet the data contains only circles and crosses. This lecture derives the cross-entropy error ln(1 + exp(−y wᵀx)) from maximum likelihood and uses the chain rule to show that the gradient is a θ-weighted average of −yₙxₙ. Setting the gradient to zero gives a non-linear equation of w with no closed-form solution, so the lecture switches to fixed learning rate gradient descent, w ← w − η∇E_in(w), and walks to the bottom of the valley one step at a time.
When no straight line separates the data, you do not need to redesign the algorithm: use a feature transform Φ to turn x into z, run a linear model in Z-space as usual, and the line in Z-space is a quadratic or higher-order curve in X-space. The Q-th order polynomial transform raises the dimension to d̃ = C(Q+d, d) − 1, and computation, storage, and the VC dimension (at most d̃+1) all grow with it, so the lecture concludes that you should try the linear model first and move to higher orders only when it is not enough.
Stepping back from a 10th-order polynomial to a 2nd-order one is the same as adding a constraint to the 10th-order problem. Once the constraint is softened to ‖w‖² ≤ C, the negative gradient of E_in at the optimal solution must be parallel to w, which gives w_REG = (ZᵀZ + λI)⁻¹Zᵀy and is the same as minimizing E_aug = E_in + (λ/N)wᵀw. That is weight decay. A larger λ means a smaller equivalent C and a smaller effective VC dimension d_EFF; switching to L1 gives sparse solutions instead.
The last Foundations lecture teaches no new algorithm. It covers three situations in which the guarantees of the first fifteen lectures stop holding. A simple model rarely separates randomly labeled data by accident: with 10 points and random labels, a decision stump separates them with probability only 20/1024, so a successful separation means something. The VC guarantee fails when training and testing do not share one distribution, and once data has affected any step of the learning process it can no longer assess the outcome. The lecture closes with Power of Three, which folds the whole course into five groups of three.
The primal hard-margin SVM has d̃+1 variables, so it stops being solvable once the feature dimension gets large. This lecture uses Lagrange duality to trade it for a quadratic program with only N variables and N+1 constraints, then uses the KKT conditions to recover w and b from α; only the support vectors, the points with α greater than 0, affect the answer. The dual's matrix still contains d̃-dimensional inner products, so the feature dimension is not really gone until the next lecture.
Hard-margin SVM insists on separating the data perfectly, so it fits noise. This lecture gives every point a violation ξₙ and uses the parameter C to trade margin width against total violation, which yields soft-margin SVM; its dual differs from the hard-margin dual only in that αₙ gains the upper bound C. The value of αₙ sorts the data into non-support vectors, free SVs, and bounded SVs, and b is computed from a free SV; C and γ are chosen by cross-validation, and the fraction of support vectors #SV/N is an upper bound on the leave-one-out error.
The representer theorem turns ridge regression into a problem in β, which gives kernel ridge regression the analytic solution β = (λI + K)⁻¹y, but almost none of the N values βₙ are 0. Replace the squared error with the ε-insensitive tube error, write the problem as a QP along the soft-margin SVM route, take the dual, and you get support vector regression (SVR): the dual has 2N variables, and every point strictly inside the tube has βₙ equal to 0.
This lecture re-reads bagging's bootstrap as putting a weight on every example, then deliberately tunes the weights so that the previous round's gₜ has a weighted error rate of exactly 1/2, which forces the next round to learn a different hypothesis. The recipe multiplies incorrect examples by ◆ₜ = √((1−εₜ)/εₜ), divides correct ones by it, and uses αₜ = ln ◆ₜ as the vote weight in a linear vote. That is AdaBoost. As long as the base algorithm is a little better than random guessing in every round, E_in drops to 0 within O(log N) rounds; with decision stumps, three hand-computed rounds on six examples are enough to see it.
A random forest is bagging applied to fully-grown C&RT trees, with each branch looking only at a few randomly projected features. When bootstrap draws N times, each example has about a 1/e ≈ 36.8% chance of not being used by a given tree. These out-of-bag examples give E_oob directly, a validation that needs no held-out data and no re-training; add the permutation test and you get an importance for every feature. The price is that the result wobbles with the random seed, so you keep adding trees until G is stable.
The sum of AdaBoost's weights in each round is exactly the sum of the exponential error exp(−ys). From that, this lecture derives that the gₜ found by the base algorithm is an approximate functional gradient direction, and the vote αₜ is the best step size along that direction. Swap the exponential error for squared error and the same derivation becomes 'in each round, run regression on the residuals yₙ − sₙ, then set the step size with one-variable linear regression'; with a weakened C&RT as the base model, that is GBDT.
Once a network is stacked deep, each layer only has to do a little, but structure, model complexity, optimization, and computation all get harder. This lecture picks two of those, initialization and regularization: pre-train the weights layer by layer with a d—d̃—d autoencoder so that each layer first learns an information-preserving encoding; train on inputs with artificial noise added, as a form of regularization; and finally make the autoencoder linear and derive that the optimal W is the d̃ top eigenvectors of XᵀX, which becomes PCA once the mean is subtracted from the data.
The only input a recommender system gets is a user ID. Apply binary vector encoding to the ID and feed it to an N-d̃-M linear network with the tanh removed, and the hypothesis is h(x) = WᵀVx, which is the same as writing the rating matrix as R ≈ VᵀW. There are two ways to learn V and W: alternating least squares solves M+N linear regressions per round, and SGD uses one rating at a time to update the vectors of one user and one movie. The lecture closes by putting boosting, neural networks, RBF networks, matrix factorization, and k nearest neighbor on one map of extraction models.
CS230 is the deep learning course Andrew Ng and Kian Katanforoosh teach at Stanford, run as a flipped classroom: the foundations of neural networks, optimization, CNNs, sequence models and Transformers live in five online Coursera courses, and one in-person lecture a week covers project method, adversarial attacks and generative models, reinforcement learning, LLM applications and interpretability. The nine in-person lectures, most of their slides and two practice midterms with solutions are public; the Stanford-only Coursera sessions, the section recordings and project mentoring are not.
The opening lecture of CS229 Spring 2026 runs only 37 minutes and has no derivations. It defines machine learning with Tom Mitchell's experience E, task T, and performance measure P, then splits the whole course into supervised, unsupervised, and reinforcement learning. The three differ in whether the data comes with answers and whether the model collected the data itself. Large language models show up in all three categories at once.