  Stochastic gradient descent samples uniformly the training set to build an
unbiased gradient estimate with a limited number of samples. However, at a
given step of the training process, some data are more helpful than others to
continue learning. Importance sampling for training deep neural networks has
been widely studied to propose sampling schemes yielding better performance
than the uniform sampling scheme. After recalling the theory of importance
sampling for deep learning, this paper reviews the challenges inherent to this
research area. In particular, we propose a metric allowing the assessment of
the quality of a given sampling scheme; and we study the interplay between the
sampling scheme and the optimizer used.
