  We propose a new self-adaptive, double-loop smoothing algorithm to solve
composite, nonsmooth, and constrained convex optimization problems. Our
algorithm is based on Nesterov's smoothing technique via general Bregman
distance functions. It self-adaptively selects the number of iterations in the
inner loop to achieve a desired complexity bound without requiring the accuracy
a priori as in variants of Augmented Lagrangian methods (ALM). We prove
$\BigO{\frac{1}{k}}$-convergence rate on the last iterate of the outer sequence
for both unconstrained and constrained settings in contrast to ergodic rates
which are common in ALM as well as alternating direction method-of-multipliers
literature. Compared to existing inexact ALM or quadratic penalty methods, our
analysis does not rely on the worst-case bounds of the subproblem solved by the
inner loop. Therefore, our algorithm can be viewed as a restarting technique
applied to the ASGARD method in \cite{TranDinh2015b} but with rigorous
theoretical guarantees or as an inexact ALM with explicit inner loop
termination rules and adaptive parameters. Our algorithm only requires to
initialize the parameters once, and automatically update them during the
iteration process without tuning. We illustrate the superiority of our methods
via several examples as compared to the state-of-the-art.
