  Neural Processes combine the strengths of neural networks and Gaussian
processes to achieve both flexible learning and fast prediction in stochastic
processes. However, a large class of problems comprises underlying temporal
dependency structures in a sequence of stochastic processes that Neural
Processes (NP) do not explicitly consider. In this paper, we propose Sequential
Neural Processes (SNP) which incorporates a temporal state-transition model of
stochastic processes and thus extends its modeling capabilities to dynamic
stochastic processes. In applying SNP to dynamic 3D scene modeling, we
introduce the Temporal Generative Query Networks. To our knowledge, this is the
first 4D model that can deal with the temporal dynamics of 3D scenes. In
experiments, we evaluate the proposed methods in dynamic (non-stationary)
regression and 4D scene inference and rendering.
