Skip to content
All tags

#em-algorithm

2 posts
aiguideReading Stanford CS229 Machine Learning

k-means and Gaussian Mixture Models: How Do You Cluster Without Labels? | Stanford CS229 Lecture 9

Lecture 9 starts from the two steps of k-means, assign and average, and explains why the algorithm always stops yet only reaches a local optimum. Replacing hard assignments with probabilities gives the Gaussian mixture model and the two steps of EM: an E-step that is Bayes' rule and an M-step that is a weighted average. The lecture ends with the picture behind EM, drawn with Jensen's inequality: each round builds a lower bound that touches the log-likelihood and climbs to its top. The full derivation waits for Lecture 10.

#cs229#ai-course#stanford#machine-learning#unsupervised-learning#k-means#em-algorithm
aiguideReading Stanford CS229 Machine Learning

The EM Algorithm and PCA: How a Lower Bound Keeps the Likelihood from Dropping | Stanford CS229 Lecture 10

The first half uses Jensen's inequality to build a lower bound on the log-likelihood, the ELBO, then picks Q to be the posterior so the bound is tight at the current parameters. From that it follows that no round of EM can lower the likelihood, and plugging in a Gaussian mixture recovers last lecture's weighted-average formulas. The second half writes PCA as maximizing uᵀΣu over unit vectors; the answer is the eigenvector of the covariance matrix with the largest eigenvalue.

#cs229#ai-course#stanford#machine-learning#em-algorithm#pca#unsupervised-learning