  Background: Deep learning techniques, particularly neural networks, have
revolutionized computational physics, offering powerful tools for solving
complex partial differential equations (PDEs). However, ensuring stability and
efficiency remains a challenge, especially in scenarios involving nonlinear and
time-dependent equations. Methodology: This paper introduces novel
residual-based architectures, namely the Simple Highway Network and the Squared
Residual Network, designed to enhance stability and accuracy in
physics-informed neural networks (PINNs). These architectures augment
traditional neural networks by incorporating residual connections, which
facilitate smoother weight updates and improve backpropagation efficiency.
Results: Through extensive numerical experiments across various examples
including linear and nonlinear, time-dependent and independent PDEs we
demonstrate the efficacy of the proposed architectures. The Squared Residual
Network, in particular, exhibits robust performance, achieving enhanced
stability and accuracy compared to conventional neural networks. These findings
underscore the potential of residual-based architectures in advancing deep
learning for PDEs and computational physics applications.
