Neural Nets & Backprop Intuition

A neural net composes differentiable layers. Training adjusts weights to reduce loss via gradient descent.

Building blocks

• Linear transforms + nonlinearities (ReLU, GELU, …)
• Softmax for multiclass outputs
• Cross-entropy loss for classification

Backprop in one sentence

Compute (\partial L / \partial w) by chain rule, efficiently via reverse-mode autodiff.