  In this paper, we propose a probabilistic optimization method, named
probabilistic incremental proximal gradient (PIPG) method, by developing a
probabilistic interpretation of the incremental proximal gradient algorithm. We
explicitly model the update rules of the incremental proximal gradient method
and develop a systematic approach to propagate the uncertainty of the solution
estimate over iterations. The PIPG algorithm takes the form of Bayesian
filtering updates for a state-space model constructed by using the cost
function. Our framework makes it possible to utilize well-known exact or
approximate Bayesian filters, such as Kalman or extended Kalman filters, to
solve large-scale regularized optimization problems.
