  Physics-informed deep learning has emerged as a promising alternative for
solving partial differential equations. However, for complex problems, training
these networks can still be challenging, often resulting in unsatisfactory
accuracy and efficiency. In this work, we demonstrate that the failure of plain
physics-informed neural networks arises from the significant discrepancy in the
convergence speed of residuals at different training points, where the slowest
convergence speed dominates the overall solution convergence. Based on these
observations, we propose a point-wise adaptive weighting method that balances
the residual decay rate across different training points. The performance of
our proposed adaptive weighting method is compared with current
state-of-the-art adaptive weighting methods on benchmark problems for both
physics-informed neural networks and physics-informed deep operator networks.
Through extensive numerical results we demonstrate that our proposed approach
of balanced residual decay rates offers several advantages, including bounded
weights, high prediction accuracy, fast convergence speed, low training
uncertainty, low computational cost and ease of hyperparameter tuning.
