  A Neural Process (NP) estimates a stochastic process implicitly defined with
neural networks given a stream of data, rather than pre-specifying priors
already known, such as Gaussian processes. An ideal NP would learn everything
from data without any inductive biases, but in practice, we often restrict the
class of stochastic processes for the ease of estimation. One such restriction
is the use of a finite-dimensional latent variable accounting for the
uncertainty in the functions drawn from NPs. Some recent works show that this
can be improved with more "data-driven" source of uncertainty such as
bootstrapping. In this work, we take a different approach based on the
martingale posterior, a recently developed alternative to Bayesian inference.
For the martingale posterior, instead of specifying prior-likelihood pairs, a
predictive distribution for future data is specified. Under specific conditions
on the predictive distribution, it can be shown that the uncertainty in the
generated future data actually corresponds to the uncertainty of the implicitly
defined Bayesian posteriors. Based on this result, instead of assuming any form
of the latent variables, we equip a NP with a predictive distribution
implicitly defined with neural networks and use the corresponding martingale
posteriors as the source of uncertainty. The resulting model, which we name as
Martingale Posterior Neural Process (MPNP), is demonstrated to outperform
baselines on various tasks.
