  In the machine learning literature stochastic gradient descent has recently
been widely discussed for its purported implicit regularization properties.
Much of the theory, that attempts to clarify the role of noise in stochastic
gradient algorithms, has widely approximated stochastic gradient descent by a
stochastic differential equation with Gaussian noise. We provide a novel
rigorous theoretical justification for this practice that showcases how the
Gaussianity of the noise arises naturally.
