  One of the open problems in scientific computing is the long-time integration
of nonlinear stochastic partial differential equations (SPDEs). We address this
problem by taking advantage of recent advances in scientific machine learning
and the dynamically orthogonal (DO) and bi-orthogonal (BO) methods for
representing stochastic processes. Specifically, we propose two new
Physics-Informed Neural Networks (PINNs) for solving time-dependent SPDEs,
namely the NN-DO/BO methods, which incorporate the DO/BO constraints into the
loss function with an implicit form instead of generating explicit expressions
for the temporal derivatives of the DO/BO modes. Hence, the proposed methods
overcome some of the drawbacks of the original DO/BO methods: we do not need
the assumption that the covariance matrix of the random coefficients is
invertible as in the original DO method, and we can remove the assumption of no
eigenvalue crossing as in the original BO method. Moreover, the NN-DO/BO
methods can be used to solve time-dependent stochastic inverse problems with
the same formulation and computational complexity as for forward problems. We
demonstrate the capability of the proposed methods via several numerical
examples: (1) A linear stochastic advection equation with deterministic initial
condition where the original DO/BO method would fail; (2) Long-time integration
of the stochastic Burgers' equation with many eigenvalue crossings during the
whole time evolution where the original BO method fails. (3) Nonlinear reaction
diffusion equation: we consider both the forward and the inverse problem,
including noisy initial data, to investigate the flexibility of the NN-DO/BO
methods in handling inverse and mixed type problems. Taken together, these
simulation results demonstrate that the NN-DO/BO methods can be employed to
effectively quantify uncertainty propagation in a wide range of physical
problems.
