  Physics-informed neural networks (PINNs) have recently become a powerful tool
for solving partial differential equations (PDEs). However, finding a set of
neural network parameters that lead to fulfilling a PDE can be challenging and
non-unique due to the complexity of the loss landscape that needs to be
traversed. Although a variety of multi-task learning and transfer learning
approaches have been proposed to overcome these issues, there is no incremental
training procedure for PINNs that can effectively mitigate such training
challenges. We propose incremental PINNs (iPINNs) that can learn multiple tasks
(equations) sequentially without additional parameters for new tasks and
improve performance for every equation in the sequence. Our approach learns
multiple PDEs starting from the simplest one by creating its own subnetwork for
each PDE and allowing each subnetwork to overlap with previously learned
subnetworks. We demonstrate that previous subnetworks are a good initialization
for a new equation if PDEs share similarities. We also show that iPINNs achieve
lower prediction error than regular PINNs for two different scenarios: (1)
learning a family of equations (e.g., 1-D convection PDE); and (2) learning
PDEs resulting from a combination of processes (e.g., 1-D reaction-diffusion
PDE). The ability to learn all problems with a single network together with
learning more complex PDEs with better generalization than regular PINNs will
open new avenues in this field.
