  This paper presents a detailed theoretical analysis of the three stochastic
approximation proximal gradient algorithms proposed in our companion paper [49]
to set regularization parameters by marginal maximum likelihood estimation. We
prove the convergence of a more general stochastic approximation scheme that
includes the three algorithms of [49] as special cases. This includes
asymptotic and non-asymptotic convergence results with natural and easily
verifiable conditions, as well as explicit bounds on the convergence rates.
Importantly, the theory is also general in that it can be applied to other
intractable optimisation problems. A main novelty of the work is that the
stochastic gradient estimates of our scheme are constructed from inexact
proximal Markov chain Monte Carlo samplers. This allows the use of samplers
that scale efficiently to large problems and for which we have precise
theoretical guarantees.
