  Recently, structure learning of directed acyclic graphs (DAGs) has been
formulated as a continuous optimization problem by leveraging an algebraic
characterization of acyclicity. The constrained problem is solved using the
augmented Lagrangian method (ALM) which is often preferred to the quadratic
penalty method (QPM) by virtue of its standard convergence result that does not
require the penalty coefficient to go to infinity, hence avoiding
ill-conditioning. However, the convergence properties of these methods for
structure learning, including whether they are guaranteed to return a DAG
solution, remain unclear, which might limit their practical applications. In
this work, we examine the convergence of ALM and QPM for structure learning in
the linear, nonlinear, and confounded cases. We show that the standard
convergence result of ALM does not hold in these settings, and demonstrate
empirically that its behavior is akin to that of the QPM which is prone to
ill-conditioning. We further establish the convergence guarantee of QPM to a
DAG solution, under mild conditions. Lastly, we connect our theoretical results
with existing approaches to help resolve the convergence issue, and verify our
findings in light of an empirical comparison of them.
