MIT 6.7960 L03: Optimization Overview — SGD, Adam, LR Schedules & Scaling Rules
From SGD to Adam: pick the right optimizer and scale LR with batch size using scaling rules
#pytorch
9 postsMIT 6.7960 L04: Regularization in Practice — Weight Decay, Dropout, Batch Norm & Label Smoothing
Regularization isn't just anti-overfitting — mechanisms & combo strategies for WD, Dropout, BN, Label Smoothing
MIT 6.7960 PS1 Walkthrough: From NumPy MLP to PyTorch Autograd Backprop
Hand-write NumPy MLP + backprop → verify with PyTorch Autograd, fully reproducing OCW HW1 core concepts
MIT 6.7960 L05: CNN Architectures — From Convolution Kernels to Translation Equivariance
Lec 4 core: why CNN is the natural choice for grid data — convolution, translation equivariance, pooling, and classic architectures in one go
MIT 6.7960 L06: Modern CNN Architectures — ResNet, EfficientNet, ConvNeXt
ResNet's skip connections solve degradation, enabling 100+ layer nets; EfficientNet compound scales depth/width/resolution; ConvNeXt absorbs Transformer design to reclaim CV crown.
MIT 6.7960 L01: Course Introduction — A Map of Deep Learning, Why Depth Works, and Your First Training Loop
Lecture 1 is the 6.7960 opener: deep learning took off because data + compute + algorithms matured together; the course threads from architectures (CNN/GNN/Transformer) through training, representation, generation, transfer, scaling, and LLMs; ends with a ~30-line PyTorch training loop to confirm your environment works.
CS336 Lecture 7: Build Data, Tensor, and Pipeline Parallelism from Collectives
Lecture 7 starts below FSDP APIs, building a communication language from broadcast, all-reduce, all-gather, reduce-scatter, and all-to-all before assembling data, tensor, and pipeline parallelism.
CS336 Lecture 2: Count FLOPs and Memory Before Asking Whether a Model Fits
Lecture 2 reduces model training to tensors, FLOPs, bytes, and time: use einops to track dimensions, arithmetic intensity and roofline analysis to identify bottlenecks, then trade compute for memory with gradient accumulation and activation checkpointing.
MIT 6.S191 Guide: Nine Lectures and Three Labs Are Public, but the Full Path Still Uses Three External Services
MIT 6.S191's 2026 edition publishes nine lecture videos, slides, three software labs, and solutions, making it an A3 self-study course. The supplied path still depends on Google/Colab, Comet, and OpenRouter for Lab 3, while unaffiliated learners do not receive MIT credit, project feedback, or API credits.