  Multi-fidelity modeling and learning are important in physical
simulation-related applications. It can leverage both low-fidelity and
high-fidelity examples for training so as to reduce the cost of data generation
while still achieving good performance. While existing approaches only model
finite, discrete fidelities, in practice, the fidelity choice is often
continuous and infinite, which can correspond to a continuous mesh spacing or
finite element length. In this paper, we propose Infinite Fidelity
Coregionalization (IFC). Given the data, our method can extract and exploit
rich information within continuous, infinite fidelities to bolster the
prediction accuracy. Our model can interpolate and/or extrapolate the
predictions to novel fidelities, which can be even higher than the fidelities
of training data. Specifically, we introduce a low-dimensional latent output as
a continuous function of the fidelity and input, and multiple it with a basis
matrix to predict high-dimensional solution outputs. We model the latent output
as a neural Ordinary Differential Equation (ODE) to capture the complex
relationships within and integrate information throughout the continuous
fidelities. We then use Gaussian processes or another ODE to estimate the
fidelity-varying bases. For efficient inference, we reorganize the bases as a
tensor, and use a tensor-Gaussian variational posterior to develop a scalable
inference algorithm for massive outputs. We show the advantage of our method in
several benchmark tasks in computational physics.
