  This paper concerns with a noisy structured low-rank matrix recovery problem
which can be modeled as a structured rank minimization problem. We reformulate
this problem as a mathematical program with a generalized complementarity
constraint (MPGCC), and show that its penalty version, yielded by moving the
generalized complementarity constraint to the objective, has the same global
optimal solution set as the MPGCC does whenever the penalty parameter is over a
threshold. Then, by solving the exact penalty problem in an alternating way, we
obtain a multi-stage convex relaxation approach. We provide theoretical
guarantees for our approach under a mild restricted eigenvalue condition, by
quantifying the reduction of the error and approximate rank bounds of the first
stage convex relaxation (which is exactly the nuclear norm relaxation) in the
subsequent stages and establishing the geometric convergence of the error
sequence in a statistical sense. Numerical experiments are conducted for some
structured low-rank matrix recovery examples to confirm our theoretical
findings.
