  Regression on function spaces is typically limited to models with Gaussian
process priors. We introduce the notion of universal functional regression, in
which we aim to learn a prior distribution over non-Gaussian function spaces
that remains mathematically tractable for functional regression. To do this, we
develop Neural Operator Flows (OpFlow), an infinite-dimensional extension of
normalizing flows. OpFlow is an invertible operator that maps the (potentially
unknown) data function space into a Gaussian process, allowing for exact
likelihood estimation of functional point evaluations. OpFlow enables robust
and accurate uncertainty quantification via drawing posterior samples of the
Gaussian process and subsequently mapping them into the data function space. We
empirically study the performance of OpFlow on regression and generation tasks
with data generated from Gaussian processes with known posterior forms and
non-Gaussian processes, as well as real-world earthquake seismograms with an
unknown closed-form distribution.
