  We consider stochastic gradient descents on the space of large symmetric
matrices of suitable functions that are invariant under permuting the rows and
columns using the same permutation. We establish deterministic limits of these
random curves as the dimensions of the matrices go to infinity while the
entries remain bounded. Under a ``small noise'' assumption the limit is shown
to be the gradient flow of functions on graphons whose existence was
established in~\cite{oh2021gradient}. We also consider limits of stochastic
gradient descents with added properly scaled reflected Brownian noise. The
limiting curve of graphons is characterized by a family of stochastic
differential equations with reflections and can be thought of as an extension
of the classical McKean-Vlasov limit for interacting diffusions to the graphon
setting. The proofs introduce a family of infinite-dimensional exchangeable
arrays of reflected diffusions and a novel notion of propagation of chaos for
large matrices of diffusions converging to such arrays in a suitable sense.
