  This paper introduces AdaSDCA: an adaptive variant of stochastic dual
coordinate ascent (SDCA) for solving the regularized empirical risk
minimization problems. Our modification consists in allowing the method
adaptively change the probability distribution over the dual variables
throughout the iterative process. AdaSDCA achieves provably better complexity
bound than SDCA with the best fixed probability distribution, known as
importance sampling. However, it is of a theoretical character as it is
expensive to implement. We also propose AdaSDCA+: a practical variant which in
our experiments outperforms existing non-adaptive methods.
