  We introduce exponential numerical integration methods for stiff stochastic
dynamical systems of the form $d\mathbf{z}_t = L(t)\mathbf{z}_tdt +
\mathbf{f}(t)dt + Q(t)d\mathbf{W}_t$. We consider the setting of time-varying
operators $L(t), Q(t)$ where they may not commute $L(t_1)L(t_2) \neq
L(t_2)L(t_1)$, raising challenges for exponentiation. We develop stochastic
numerical integration methods using Mangus expansions for preserving
statistical structures and for maintaining fluctuation-dissipation balance for
physical systems. For computing the contributions of the fluctuation terms, our
methods provide alternative approaches without needing directly to evaluate
stochastic integrals. We present results for our methods for a class of SDEs
arising in particle simulations and for SPDEs for fluctuations of concentration
fields in spatially-extended systems. For time-varying stochastic dynamical
systems, our introduced discretization approaches provide general exponential
numerical integrators for preserving statistical structures while handling
stiffness.
