  In this paper, we propose a pre-trained foundation model \textbf{FMint}
(\textbf{F}oundation \textbf{M}odel based on \textbf{In}i\textbf{t}ialization),
designed to speed up large-scale simulations of various differential equations
with high accuracy via error correction. Human-designed simulation algorithms
excel at capturing the fundamental physics of engineering problems, but often
need to balance the trade-off between accuracy and efficiency. While deep
learning methods offer innovative solutions across numerous scientific fields,
they frequently fall short in domain-specific knowledge. FMint bridges these
gaps through conditioning on the initial coarse solutions obtained from
conventional human-designed algorithms, and trained to obtain refined solutions
for various differential equations. Based on the backbone of large language
models, we adapt the in-context learning scheme to learn a universal error
correction method for dynamical systems from given prompted sequences of coarse
solutions. The model is pre-trained on a corpus of 600K ordinary differential
equations (ODEs), and we conduct extensive experiments on both in-distribution
and out-of-distribution tasks. FMint outperforms various baselines on
large-scale simulation, and demonstrates its capability in generalization to
unseen ODEs. Our approach achieves an accuracy improvement of 1 to 2 orders of
magnitude over state-of-the-art dynamical system simulators, and delivers a 5X
speedup compared to traditional numerical algorithms.
