  In high dimensional regression settings, sparsity enforcing penalties have
proved useful to regularize the data-fitting term. A recently introduced
technique called screening rules propose to ignore some variables in the
optimization leveraging the expected sparsity of the solutions and consequently
leading to faster solvers. When the procedure is guaranteed not to discard
variables wrongly the rules are said to be safe. In this work, we propose a
unifying framework for generalized linear models regularized with standard
sparsity enforcing penalties such as $\ell_1$ or $\ell_1/\ell_2$ norms. Our
technique allows to discard safely more variables than previously considered
safe rules, particularly for low regularization parameters. Our proposed Gap
Safe rules (so called because they rely on duality gap computation) can cope
with any iterative solver but are particularly well suited to (block)
coordinate descent methods. Applied to many standard learning tasks, Lasso,
Sparse-Group Lasso, multi-task Lasso, binary and multinomial logistic
regression, etc., we report significant speed-ups compared to previously
proposed safe rules on all tested data sets.
