Assignment 3 in CS231N Spring 2026 is worth 15% of the grade and turns L8, L12–L14 and L16 into four Colab notebooks. Q1 has you write multi-head attention and a Transformer decoder for COCO captioning, then assemble a ViT and train it on CIFAR-10. Q2 implements SimCLR's augmentations and contrastive loss and compares linear classification with and without self-supervised pretraining. Q3 builds DDPM's noising, UNet, denoising loss, sampling and classifier-free guidance to generate text-conditioned 32×32 emoji. Q4 uses pretrained CLIP for similarity, zero-shot classification and retrieval, then segments a video with DINO features trained on a single labeled frame. This guide covers structure, files and targets only. No solutions.
CS231N Lecture 12 asks whether we can learn good representations without huge manually labeled datasets. The answer comes in three parts. First, pretext tasks that generate labels from image transformations: predicting rotation, solving jigsaw puzzles, inpainting, colorization, and MAE with a 75% mask ratio. Second, the more general contrastive learning: the InfoNCE loss, SimCLR with its large batches, MoCo, which decouples batch size from the number of negatives with a queue, and sequence-level CPC. Third, DINO, which needs no negatives: a student predicts the output of a momentum teacher, and centering plus sharpening prevent collapse. The core evaluation is linear probing: freeze the encoder and train only a linear classifier.
A lot of data does not mean a lot of annotation. This lecture asks: with no annotations, how do you learn useful representations from data? The shared idea is to find the latent factors behind the observations. An auto-encoder compresses the input into a short code and reconstructs it. A VAE additionally requires the code to follow a Gaussian distribution, so it can generate by sampling from the distribution. Dual learning lets two tasks that run in opposite directions serve as each other's feedback. Self-supervised learning divides into self-prediction (hide a part and guess it back) and contrastive learning (pull similar ones together, push dissimilar ones apart). CLIP does contrastive learning on image-text pairs, so image classification needs no separate training; DALL·E 2 then uses its representations to generate images. The lecture ends with multimodal models: projecting images into the language model's space, and streaming interaction that listens while deciding when to speak.
HW5 is the only Spring 2026 assignment marked optional, due 5/11, the same day as the final. The written part has three pieces: derive InfoNCE gradients and the trade-off in the number of negatives, using scRNA-seq as the setting; prove the optimal denoiser is a conditional expectation and derive the continuity equation; generalize flow matching's straight-line path to arbitrary interpolations. The notebook is a full LLM fine-tuning pipeline: Qwen2.5-0.5B-Instruct is fixed, MMLU machine_learning is converted to chat format, TRL's SFTTrainer does full fine-tuning, accuracy on CS189 exam questions is compared before and after, and predictions on a 169-question test set go to Kaggle, all while guarding against catastrophic forgetting. An official hw5-sol.pdf is provided, covering the written part only.
Lec 23 wires a transformer into a next-token predictor: tokenize, look up embeddings, stack L layers of masked attention, multiply back by the embedding table and apply softmax, and train with cross-entropy (that is, MLE). Pretraining supplies knowledge; to chat, a model also needs SFT, LoRA, RLHF, or DPO, and at inference time it leans on in-context learning, RAG, chain-of-thought, and tool calls. Lec 24 generalizes "invent a fake supervised task" to images: autoencoders, colorization, inpainting, rotation, jigsaw puzzles, clustering, and finally contrastive learning, SimCLR, and CLIP. Discussion 11 practices positional encodings, RoPE, causal masks, and the KV cache.
HW5 Problems 1–2 both turn learning into a classification task. SimCLR's NT-Xent loss asks the network to pick the other augmented view of the same image out of 2N−1 candidates; a GAN's discriminator classifies real versus fake. You derive the math behind each (the cross-entropy equivalence, the optimal discriminator and the JS divergence), then write both training loops on FashionMNIST and MNIST.
2021 was the year diffusion models surpassed GANs, self-supervised learning made theoretical breakthroughs, and reinforcement learning confronted weaknesses in its evaluation methodology. NeurIPS received a then-record 9,122 submissions, ICLR’s Score-Based Generative Modeling paper became a theoretical foundation for the diffusion ecosystem, and ICML delivered substantial work on optimization theory and the dynamics of self-supervised learning.
2021 was a dividing line for major AI conferences. Transformers spread from NLP throughout computer vision and time-series research, self-supervised learning became the most common cross-conference theme, and a diffusion model won an ICLR Outstanding Paper award before anyone realized it would displace GANs. Meanwhile, GNNs and federated learning reached historic peaks in paper volume before beginning to decline.
CS230's second lecture derives embeddings through three case studies: day/night classification teaches you to use humans as a proxy for choosing resolution, trigger-word detection teaches you to manufacture a million training examples in three hours, and face verification walks you through designing your first loss function. The final step — from supervised triplets to self-supervised pairs — is why modern models can consume billions of unlabeled images.