← Gradient Descent — Topic Deep Dive
Free
Optimizers in Practice
From SGD to AdamW with schedules that work.
Cheatsheet — Optimizers
| Optimizer | Notes |
|---|---|
| SGD+momentum | Great generalization often |
| AdamW | Fast start; watch generalization |
| Warmup | Stabilizes large-batch / transformers |
Log LR, grad norm, and weight norm.