  Physics-Informed Neural Networks (PINNs) have become a prominent application
of deep learning in scientific computation, as they are powerful approximators
of solutions to nonlinear partial differential equations (PDEs). There have
been numerous attempts to facilitate the training process of PINNs by adjusting
the weight of each component of the loss function, called adaptive
loss-balancing algorithms. In this paper, we propose an Augmented Lagrangian
relaxation method for PINNs (AL-PINNs). We treat the initial and boundary
conditions as constraints for the optimization problem of the PDE residual. By
employing Augmented Lagrangian relaxation, the constrained optimization problem
becomes a sequential max-min problem so that the learnable parameters $\lambda$
adaptively balance each loss component. Our theoretical analysis reveals that
the sequence of minimizers of the proposed loss functions converges to an
actual solution for the Helmholtz, viscous Burgers, and Klein--Gordon
equations. We demonstrate through various numerical experiments that AL-PINNs
yield a much smaller relative error compared with that of state-of-the-art
adaptive loss-balancing algorithms.
