  This is a handbook of simple proofs of the convergence of gradient and
stochastic gradient descent type methods. We consider functions that are
Lipschitz, smooth, convex, strongly convex, and/or Polyak-{\L}ojasiewicz
functions. Our focus is on ``good proofs'' that are also simple. Each section
can be consulted separately. We start with proofs of gradient descent, then on
stochastic variants, including minibatching and momentum. Then move on to
nonsmooth problems with the subgradient method, the proximal gradient descent
and their stochastic variants. Our focus is on global convergence rates and
complexity rates. Some slightly less common proofs found here include that of
SGD (Stochastic gradient descent) with a proximal step, with momentum, and with
mini-batching without replacement.
