Optimization Overview — SGD, Adam, LR Schedules & Scaling Rules | MIT 6.7960 Topic Guide
From SGD to Adam: pick the right optimizer and scale LR with batch size using scaling rules
From SGD to Adam: pick the right optimizer and scale LR with batch size using scaling rules