MIT 6.7960 L03: Optimization Overview — SGD, Adam, LR Schedules & Scaling Rules
From SGD to Adam: pick the right optimizer and scale LR with batch size using scaling rules
From SGD to Adam: pick the right optimizer and scale LR with batch size using scaling rules