  A multi-layer deep Gaussian process (DGP) model is a hierarchical composition
of GP models with a greater expressive power. Exact DGP inference is
intractable, which has motivated the recent development of deterministic and
stochastic approximation methods. Unfortunately, the deterministic
approximation methods yield a biased posterior belief while the stochastic one
is computationally costly. This paper presents an implicit posterior
variational inference (IPVI) framework for DGPs that can ideally recover an
unbiased posterior belief and still preserve time efficiency. Inspired by
generative adversarial networks, our IPVI framework achieves this by casting
the DGP inference problem as a two-player game in which a Nash equilibrium,
interestingly, coincides with an unbiased posterior belief. This consequently
inspires us to devise a best-response dynamics algorithm to search for a Nash
equilibrium (i.e., an unbiased posterior belief). Empirical evaluation shows
that IPVI outperforms the state-of-the-art approximation methods for DGPs.
