  We propose a generic variance-reduced algorithm, which we call MUltiple
RANdomized Algorithm (MURANA), for minimizing a sum of several smooth functions
plus a regularizer, in a sequential or distributed manner. Our method is
formulated with general stochastic operators, which allow us to model various
strategies for reducing the computational complexity. For example, MURANA
supports sparse activation of the gradients, and also reduction of the
communication load via compression of the update vectors. This versatility
allows MURANA to cover many existing randomization mechanisms within a unified
framework, which also makes it possible to design new methods as special cases.
