  Neural Processes (NPs) are a class of models that learn a mapping from a
context set of input-output pairs to a distribution over functions. They are
traditionally trained using maximum likelihood with a KL divergence
regularization term. We show that there are desirable classes of problems where
NPs, with this loss, fail to learn any reasonable distribution. We also show
that this drawback is solved by using approximations of Wasserstein distance
which calculates optimal transport distances even for distributions of disjoint
support. We give experimental justification for our method and demonstrate
performance. These Wasserstein Neural Processes (WNPs) maintain all of the
benefits of traditional NPs while being able to approximate a new class of
function mappings.
