  For multi-scale problems, the conventional physics-informed neural networks
(PINNs) face some challenges in obtaining available predictions. In this paper,
based on PINNs, we propose a practical deep learning framework for multi-scale
problems by reconstructing the loss function and associating it with special
neural network architectures. New PINN methods derived from the improved PINN
framework differ from the conventional PINN method mainly in two aspects.
First, the new methods use a novel loss function by modifying the standard loss
function through a (grouping) regularization strategy. The regularization
strategy implements a different power operation on each loss term so that all
loss terms composing the loss function are of approximately the same order of
magnitude, which makes all loss terms be optimized synchronously during the
optimization process. Second, for the multi-frequency or high-frequency
problems, in addition to using the modified loss function, new methods upgrade
the neural network architecture from the common fully-connected neural network
to special network architectures such as the Fourier feature architecture, and
the integrated architecture developed by us. The combination of the above two
techniques leads to a significant improvement in the computational accuracy of
multi-scale problems. Several challenging numerical examples demonstrate the
effectiveness of the proposed methods. The proposed methods not only
significantly outperform the conventional PINN method in terms of computational
efficiency and computational accuracy, but also compare favorably with the
state-of-the-art methods in the recent literature. The improved PINN framework
facilitates better application of PINNs to multi-scale problems.
