Gradient descent is the optimization algorithm that trains neural networks: measure how wrong the model is (loss), compute which direction each weight should mo
The optimization algorithm that trains neural networks: measure how wrong the model is (loss), compute which direction each weight should move to reduce the error (the gradient), step all weights slightly that way, and repeat billions of times.
Variants like Adam and its relatives, plus learning-rate schedules, are the practical knobs of training craft. Every modern model, from a small classifier to a frontier LLM, was shaped by this loop.