  We give a sharp convergence rate for the asynchronous stochastic gradient
descent (ASGD) algorithms when the loss function is a perturbed quadratic
function based on the stochastic modified equations introduced in [An et al.
Stochastic modified equations for the asynchronous stochastic gradient descent,
arXiv:1805.08244]. We prove that when the number of local workers is larger
than the expected staleness, then ASGD is more efficient than stochastic
gradient descent. Our theoretical result also suggests that longer delays
result in slower convergence rate. Besides, the learning rate cannot be smaller
than a threshold inversely proportional to the expected staleness.
