  Gradient descent methods and especially their stochastic variants have become
highly popular in the last decade due to their efficiency on big data
optimization problems. In this thesis we present the development of data
sampling strategies for these methods. In the first four chapters we focus on
four views on the sampling for convex problems, developing and analyzing new
state-of-the-art methods using non-standard data sampling strategies. Finally,
in the last chapter we present a more flexible framework, which generalizes to
more problems as well as more sampling rules.
