  The standard approaches to neural network implementation yield powerful
function approximation capabilities but are limited in their abilities to learn
meta representations and reason probabilistic uncertainties in their
predictions. Gaussian processes, on the other hand, adopt the Bayesian learning
scheme to estimate such uncertainties but are constrained by their efficiency
and approximation capacity. The Neural Processes Family (NPF) intends to offer
the best of both worlds by leveraging neural networks for meta-learning
predictive uncertainties. Such potential has brought substantial research
activity to the family in recent years. Therefore, a comprehensive survey of
NPF models is needed to organize and relate their motivation, methodology, and
experiments. This paper intends to address this gap while digging deeper into
the formulation, research themes, and applications concerning the family
members. We shed light on their potential to bring several recent advances in
other deep learning domains under one umbrella. We then provide a rigorous
taxonomy of the family and empirically demonstrate their capabilities for
modeling data generating functions operating on 1-d, 2-d, and 3-d input
domains. We conclude by discussing our perspectives on the promising directions
that can fuel the research advances in the field. Code for our experiments will
be made available at https://github.com/srvCodes/neural-processes-survey.
