  Deep latent variable models have achieved significant empirical successes in
model-based reinforcement learning (RL) due to their expressiveness in modeling
complex transition dynamics. On the other hand, it remains unclear
theoretically and empirically how latent variable models may facilitate
learning, planning, and exploration to improve the sample efficiency of RL. In
this paper, we provide a representation view of the latent variable models for
state-action value functions, which allows both tractable variational learning
algorithm and effective implementation of the optimism/pessimism principle in
the face of uncertainty for exploration. In particular, we propose a
computationally efficient planning algorithm with UCB exploration by
incorporating kernel embeddings of latent variable models. Theoretically, we
establish the sample complexity of the proposed approach in the online and
offline settings. Empirically, we demonstrate superior performance over current
state-of-the-art algorithms across various benchmarks.
