  We consider two classes of natural stochastic processes on finite unlabeled
graphs. These are Euclidean stochastic optimization algorithms on the adjacency
matrix of weighted graphs and a modified version of the Metropolis MCMC
algorithm on stochastic block models over unweighted graphs. In both cases we
show that, as the size of the graph goes to infinity, the random trajectories
of the stochastic processes converge to deterministic curves on the space of
measure-valued graphons. Measure-valued graphons, introduced by Lov\'{a}sz and
Szegedy in \cite{lovasz2010decorated}, are a refinement of the concept of
graphons that can distinguish between two infinite exchangeable arrays that
give rise to the same graphon limit. We introduce new metrics on this space
which provide us with a natural notion of convergence for our limit theorems.
This notion is equivalent to the convergence of infinite-exchangeable arrays.
Under suitable assumptions and a specified time-scaling, the Metropolis chain
admits a diffusion limit as the number of vertices go to infinity. We then
demonstrate that, in an appropriately formulated zero-noise limit, the
stochastic process of adjacency matrices of this diffusion converges to a
deterministic gradient flow curve on the space of graphons introduced
in\cite{Oh2023}. A novel feature of this approach is that it provides a precise
exponential convergence rate for the Metropolis chain in a certain limiting
regime. The connection between a natural Metropolis chain commonly used in
exponential random graph models and gradient flows on graphons, to the best of
our knowledge, is new in the literature as well.
