  While the popularity of physics-informed neural networks (PINNs) is steadily
rising, to this date, PINNs have not been successful in simulating multi-scale
and singular perturbation problems. In this work, we present a new training
paradigm referred to as "gradient boosting" (GB), which significantly enhances
the performance of physics informed neural networks (PINNs). Rather than
learning the solution of a given PDE using a single neural network directly,
our algorithm employs a sequence of neural networks to achieve a superior
outcome. This approach allows us to solve problems presenting great challenges
for traditional PINNs. Our numerical experiments demonstrate the effectiveness
of our algorithm through various benchmarks, including comparisons with finite
element methods and PINNs. Furthermore, this work also unlocks the door to
employing ensemble learning techniques in PINNs, providing opportunities for
further improvement in solving PDEs.
