  Iterative regularization exploits the implicit bias of an optimization
algorithm to regularize ill-posed problems. Constructing algorithms with such
built-in regularization mechanisms is a classic challenge in inverse problems
but also in modern machine learning, where it provides both a new perspective
on algorithms analysis, and significant speed-ups compared to explicit
regularization. In this work, we propose and study the first iterative
regularization procedure able to handle biases described by non smooth and non
strongly convex functionals, prominent in low-complexity regularization. Our
approach is based on a primal-dual algorithm of which we analyze convergence
and stability properties, even in the case where the original problem is
unfeasible. The general results are illustrated considering the special case of
sparse recovery with the $\ell_1$ penalty. Our theoretical results are
complemented by experiments showing the computational benefits of our approach.
