This lecture first shows that the data D alone cannot determine the answer outside D (No Free Lunch), then adds one assumption: the data and the future inputs are drawn independently from the same distribution. Hoeffding's inequality then guarantees that, for a hypothesis h fixed in advance, the probability that E_in and E_out differ by more than ε is at most 2exp(−2ε²N). An algorithm has to choose among M hypotheses, so the bound is multiplied by M and becomes 2M exp(−2ε²N); with finitely many hypotheses, enough data, and a small E_in, learning is feasible.
CS189 Spring 2026 HW1 has three pieces: ten written math warm-ups (a linear system, limits of matrix powers via eigendecomposition, SVD, a matrix that flips an image, partial derivatives, a chain rule over a recursion, and four probability problems including Bayes for cancer screening), plus two public Modal notebooks on Fashion-MNIST. Part 1 drills pandas / Plotly / K-means / an MLP / matrix-based image augmentation / tensor puzzles; Part 2 does price regression, MAE / MSE / R², confusion matrices, and finally a secret test set whose images have been rotated. Due Feb 20; no official solutions.
Lectures 4–7 of CS189 Spring 2026 tie unsupervised learning into one thread. K-means clusters the data, then its weaknesses show up: hard assignments, no probabilistic framing, and a bias toward round clusters of similar size. The course reviews probability, introduces maximum likelihood estimation (MLE) and multivariate Gaussians, and rewrites K-means as a Gaussian mixture model (GMM). The GMM log-likelihood has no closed-form solution, and that gap leads the course to gradient descent. All four lectures have slides and video; Discussions 2–3 come with solutions and walkthroughs.
Lecture 8 swaps MLE's argmax p(D|θ) for argmax p(θ|D). The prior p(θ) multiplies the likelihood, and after taking the negative log it becomes an extra term in the objective. A trick coin shows data overwhelming the prior, a Beta prior estimates a click rate, and Gaussian and Laplace priors on linear-regression weights turn into L2 and L1 regularization.
The Lec10 slides are not on the 07-380 course site yet, so this guide uses only the PR6 Bayes Nets pre-reading and the 15-281 Bayes Net Demo. A joint distribution can answer any query, but nobody hands it to you and it is too big to store; a Bayes net writes it as a product of 'node given parents' tables, and every missing edge is an independence assumption.
Week 2 is about probability: conditional probability and Bayes' rule turn evidence into probabilities; a Bayesian network uses arrows to split a joint probability into one conditional probability table per node, inference by enumeration is exact but slow, and rejection sampling and likelihood weighting approximate it with samples; a Markov chain describes a state that changes over time with a transition model, and a hidden Markov model adds a sensor model to work back from observations to states you cannot see. With time links for every section, plus the approach and checks for the Heredity and PageRank projects.
HW0 checks CS181 prerequisites in four problems — y=Xw solvability, optimizing an objective, reasoning about randomness, and OLS in Python. The problem that slows you down most is the gap to patch before HW1.
Lecture 16 starts from likelihood p(D|θ), uses i.i.d. to factor the joint probability and logs to turn products into sums; Bernoulli MLE yields sample proportions, conditional Bernoulli yields logistic cross-entropy, and Gaussian noise yields squared error.
Expectation compresses a distribution into a weighted average; LOTUS handles transformed values, while linearity makes sums tractable even without independence.
A continuous variable assigns zero probability to a point and area to intervals; CDFs, Uniform, Exponential, and memorylessness build on that distinction.
Standardization maps Normal variables to Z; Phi, linear transforms, and continuity correction turn intervals and large binomials into computable probabilities.
A joint distribution retains the full relationship among variables; marginals, conditionals, independence, and Bayes extract different answers from it.
Inference multiplies each hidden-variable prior by an observation likelihood and normalizes; the same loop handles repeated evidence and discretized continuous beliefs.
A Bayesian network factorizes a huge joint through conditional independence; ancestral sampling generates joint samples, and rejection sampling filters them into a conditional.
The Multinomial extends two-category Binomial counts to many categories; the same PMF models documents as word counts for Bayesian authorship with log-scores.
A Beta distribution represents full belief about an unknown success rate; success/failure data updates two parameters for posteriors, smoothing, and Thompson-sampling decisions.
The bootstrap treats a sample histogram as a population proxy, resampling with replacement to approximate a statistic's sampling distribution, error bar, or null p-value.
Expected cost in randomized code can be conditioned on the first random choice; counting problems become indicator sums, often avoiding the full distribution entirely.
Surprise turns rare events into bits; entropy is expected surprise, information gain selects uncertainty-reducing questions, and KL measures excess cost from a model distribution.
Logistic regression turns a linear score into a Bernoulli probability with sigmoid; the gradient xⱼ(y-ŷ) follows directly from the log-likelihood chain rule.
CS109 is Stanford's probability course for computer scientists: it runs from counting, conditional probability and random variables to inference, maximum likelihood estimation, logistic regression and deep learning. The interactive course reader and the in-class worksheets with answer keys are public, and every lecture comes with an official LLM learning guide; slides after Lecture 15, the problem-set system and the recordings are closed to outsiders.