  First-order methods (FOMs) have been widely used for solving large-scale
problems. A majority of existing works focus on problems without constraint or
with simple constraints. Several recent works have studied FOMs for problems
with complicated functional constraints. In this paper, we design a novel
augmented Lagrangian (AL) based FOM for solving problems with non-convex
objective and convex constraint functions. The new method follows the framework
of the proximal point (PP) method. On approximately solving PP subproblems, it
mixes the usage of the inexact AL method (iALM) and the quadratic penalty
method, while the latter is always fed with estimated multipliers by the iALM.
We show a complexity result of $O(\varepsilon^{-\frac{5}{2}}|\log\varepsilon|)$
for the proposed method to achieve an $\varepsilon$-KKT point. This is the best
known result. Theoretically, the hybrid method has lower iteration-complexity
requirement than its counterpart that only uses iALM to solve PP subproblems,
and numerically, it can perform significantly better than a pure-penalty-based
method. Numerical experiments are conducted on nonconvex linearly constrained
quadratic programs and nonconvex QCQP. The numerical results demonstrate the
efficiency of the proposed methods over existing ones.
