  The article discusses distributed gradient-descent algorithms for computing
local and global minima in nonconvex optimization. For local optimization, we
focus on distributed stochastic gradient descent (D-SGD)--a simple
network-based variant of classical SGD. We discuss local minima convergence
guarantees and explore the simple but critical role of the stable-manifold
theorem in analyzing saddle-point avoidance. For global optimization, we
discuss annealing-based methods in which slowly decaying noise is added to
D-SGD. Conditions are discussed under which convergence to global minima is
guaranteed. Numerical examples illustrate the key concepts in the paper.
