What is Probability?: List outcomes first; only then assign probabilities to events. | Stanford CS109 Lecture 1
List outcomes first; only then assign probabilities to events.
#cs109
23 postsConditional Probability: A condition restricts the sample space to outcomes still compatible with the evidence. | Stanford CS109 Lecture 2
A condition restricts the sample space to outcomes still compatible with the evidence.
Bayes Theorem: Bayes’ theorem turns an easier generative direction into the inferential direction we need. | Stanford CS109 Lecture 3
Bayes’ theorem turns an easier generative direction into the inferential direction we need.
Counting and Combinatorics: Decide whether order matters and repetition is allowed before choosing a formula. | Stanford CS109 Lecture 4
Decide whether order matters and repetition is allowed before choosing a formula.
Random Variables and Expectation: A random variable maps outcomes to numbers; expectation is a weighted average, not necessarily an attainable value. | Stanford CS109 Lecture 5
A random variable maps outcomes to numbers; expectation is a weighted average, not necessarily an attainable value.
Moments: Expectation, LOTUS, and linearity | Stanford CS109 Lecture 6
Expectation compresses a distribution into a weighted average; LOTUS handles transformed values, while linearity makes sums tractable even without independence.
Variance and Poisson: From spread to rare-event counts | Stanford CS109 Lecture 7
Variance describes a random variable's spread; Poisson models counts in a fixed interval and approximates a large-n, small-p binomial.
Continuous Random Variables: PDFs, CDFs, Uniform, and Exponential | Stanford CS109 Lecture 8
A continuous variable assigns zero probability to a point and area to intervals; CDFs, Uniform, Exponential, and memorylessness build on that distinction.
Normal Distribution: Standardization, Phi, and continuity correction | Stanford CS109 Lecture 9
Standardization maps Normal variables to Z; Phi, linear transforms, and continuity correction turn intervals and large binomials into computable probabilities.
Probabilistic Models: Joints, marginals, independence, and Bayes | Stanford CS109 Lecture 10
A joint distribution retains the full relationship among variables; marginals, conditionals, independence, and Bayes extract different answers from it.
Inference: Prior times likelihood, then normalize | Stanford CS109 Lecture 11
Inference multiplies each hidden-variable prior by an observation likelihood and normalizes; the same loop handles repeated evidence and discretized continuous beliefs.
General Inference: Bayesian networks, sampling, and rare evidence | Stanford CS109 Lecture 12
A Bayesian network factorizes a huge joint through conditional independence; ancestral sampling generates joint samples, and rejection sampling filters them into a conditional.
Multinomial: Category counts, bag of words, and log probability | Stanford CS109 Lecture 13
The Multinomial extends two-category Binomial counts to many categories; the same PMF models documents as word counts for Bayesian authorship with log-scores.
Beta: Turn an unknown probability into an updatable random variable | Stanford CS109 Lecture 14
A Beta distribution represents full belief about an unknown success rate; success/failure data updates two parameters for posteriors, smoothing, and Thompson-sampling decisions.
Adding Random Variables and the Central Limit Theorem | Stanford CS109 Lecture 15
A few independent sums have closed forms; general IID sums become approximately Normal under the CLT, with continuity correction for discrete sums.
Bootstrapping: Sampling statistics, error bars, and p-values | Stanford CS109 Lecture 16
The bootstrap treats a sample histogram as a population proxy, resampling with replacement to approximate a statistic's sampling distribution, error bar, or null p-value.
Algorithmic Analysis: Conditional expectation, indicators, and recursion | Stanford CS109 Lecture 17
Expected cost in randomized code can be conditioned on the first random choice; counting problems become indicator sums, often avoiding the full distribution entirely.
Information Theory: Surprise, entropy, information gain, and KL | Stanford CS109 Lecture 18
Surprise turns rare events into bits; entropy is expected surprise, information gain selects uncertainty-reducing questions, and KL measures excess cost from a model distribution.
Maximum Likelihood Estimation: Hold data fixed and optimize the parameter | Stanford CS109 Lecture 19
MLE fixes observed data and optimizes parameters; log-likelihood turns products into sums, but a maximum can also lie on a boundary.
Logistic Regression: Derive the gradient from Bernoulli likelihood | Stanford CS109 Lecture 20
Logistic regression turns a linear score into a Bernoulli probability with sigmoid; the gradient xⱼ(y-ŷ) follows directly from the log-likelihood chain rule.
Comparing Classifiers: Beyond accuracy to calibration, error costs, and fairness | Stanford CS109 Lecture 21
Classifier comparison requires held-out data, baselines, calibration, precision/recall, and an explicit fairness criterion—not accuracy alone.
Deep Learning: Derive backpropagation with the chain rule | Stanford CS109 Lecture 22
A neural network stacks logistic units; a forward pass computes probabilities, while backpropagation reuses output error to obtain every gradient.
Probability for Computer Scientists: From Counting and Inference to Machine Learning | Stanford CS109 Course Overview
CS109 is Stanford's probability course for computer scientists: it runs from counting, conditional probability and random variables to inference, maximum likelihood estimation, logistic regression and deep learning. The interactive course reader and the in-class worksheets with answer keys are public, and every lecture comes with an official LLM learning guide; slides after Lecture 15, the problem-set system and the recordings are closed to outsiders.