  Augmented Lagrangian method (ALM) has been popularly used for solving
constrained optimization problems. Practically, subproblems for updating primal
variables in the framework of ALM usually can only be solved inexactly. The
convergence and local convergence speed of ALM have been extensively studied.
However, the global convergence rate of inexact ALM is still open for problems
with nonlinear inequality constraints. In this paper, we work on general convex
programs with both equality and inequality constraints. For these problems, we
establish the global convergence rate of inexact ALM and estimate its iteration
complexity in terms of the number of gradient evaluations to produce a solution
with a specified accuracy.
  We first establish an ergodic convergence rate result of inexact ALM that
uses constant penalty parameters or geometrically increasing penalty
parameters. Based on the convergence rate result, we apply Nesterov's optimal
first-order method on each primal subproblem and estimate the iteration
complexity of the inexact ALM. We show that if the objective is convex, then
$O(\varepsilon^{-1})$ gradient evaluations are sufficient to guarantee an
$\varepsilon$-optimal solution in terms of both primal objective and
feasibility violation. If the objective is strongly convex, the result can be
improved to $O(\varepsilon^{-\frac{1}{2}}|\log\varepsilon|)$. Finally, by
relating to the inexact proximal point algorithm, we establish a nonergodic
convergence rate result of inexact ALM that uses geometrically increasing
penalty parameters. We show that the nonergodic iteration complexity result is
in the same order as that for the ergodic result. Numerical experiments on
quadratically constrained quadratic programming are conducted to compare the
performance of the inexact ALM with different settings.
