  We introduce a novel two-level overlapping additive Schwarz preconditioner
for accelerating the training of scientific machine learning applications. The
design of the proposed preconditioner is motivated by the nonlinear two-level
overlapping additive Schwarz preconditioner. The neural network parameters are
decomposed into groups (subdomains) with overlapping regions. In addition, the
network's feed-forward structure is indirectly imposed through a novel
subdomain-wise synchronization strategy and a coarse-level training step.
Through a series of numerical experiments, which consider physics-informed
neural networks and operator learning approaches, we demonstrate that the
proposed two-level preconditioner significantly speeds up the convergence of
the standard (LBFGS) optimizer while also yielding more accurate machine
learning models. Moreover, the devised preconditioner is designed to take
advantage of model-parallel computations, which can further reduce the training
time.
