  We study a class of graphon particle systems with time-varying random
coefficients. In a graphon particle system, the interactions among particles
are characterized by the coupled mean field terms through an underlying graphon
and the randomness of the coefficients comes from the stochastic processes
associated with the particle labels. By constructing two-level approximated
sequences converging in 2-Wasserstein distance, we prove the existence and
uniqueness of the solution to the system. Besides, by constructing two-level
approximated functions converging to the graphon mean field terms, we establish
the law of large numbers, which reveals that if the number of particles tends
to infinity and the discretization step tends to zero, then the discrete-time
interacting particle system over a large-scale network converges to the graphon
particle system. As a byproduct, we discover that the graphon particle system
can describe the limiting dynamics of the distributed stochastic gradient
descent algorithm over the large-scale network and prove that if the gradients
of the local cost functions are Lipschitz continuous, then the graphon particle
system can be regarded as the spatio-temporal approximation of the
discrete-time distributed stochastic gradient descent algorithm as the number
of network nodes tends to infinity and the algorithm step size tends to zero.
