  Optimization of conflicting functions is of paramount importance in decision
making, and real world applications frequently involve data that is uncertain
or unknown, resulting in multi-objective optimization (MOO) problems of
stochastic type. We study the stochastic multi-gradient (SMG) method, seen as
an extension of the classical stochastic gradient method for single-objective
optimization.
  At each iteration of the SMG method, a stochastic multi-gradient direction is
calculated by solving a quadratic subproblem, and it is shown that this
direction is biased even when all individual gradient estimators are unbiased.
We establish rates to compute a point in the Pareto front, of order similar to
what is known for stochastic gradient in both convex and strongly convex cases.
The analysis handles the bias in the multi-gradient and the unknown a priori
weights of the limiting Pareto point.
  The SMG method is framed into a Pareto-front type algorithm for the
computation of the entire Pareto front. The Pareto-front SMG algorithm is
capable of robustly determining Pareto fronts for a number of synthetic test
problems. One can apply it to any stochastic MOO problem arising from
supervised machine learning, and we report results for logistic binary
classification where multiple objectives correspond to distinct-sources data
groups.
