  We prove that the empirical risk of most well-known loss functions factors
into a linear term aggregating all labels with a term that is label free, and
can further be expressed by sums of the loss. This holds true even for
non-smooth, non-convex losses and in any RKHS. The first term is a (kernel)
mean operator --the focal quantity of this work-- which we characterize as the
sufficient statistic for the labels. The result tightens known generalization
bounds and sheds new light on their interpretation.
  Factorization has a direct application on weakly supervised learning. In
particular, we demonstrate that algorithms like SGD and proximal methods can be
adapted with minimal effort to handle weak supervision, once the mean operator
has been estimated. We apply this idea to learning with asymmetric noisy
labels, connecting and extending prior work. Furthermore, we show that most
losses enjoy a data-dependent (by the mean operator) form of noise robustness,
in contrast with known negative results.
