  Sample efficiency has been one of the major challenges for deep reinforcement
learning. Recently, model-based reinforcement learning has been proposed to
address this challenge by performing planning on imaginary trajectories with a
learned world model. However, world model learning may suffer from overfitting
to training trajectories, and thus model-based value estimation and policy
search will be pone to be sucked in an inferior local policy. In this paper, we
propose a novel model-based reinforcement learning algorithm, called BrIdging
Reality and Dream (BIRD). It maximizes the mutual information between imaginary
and real trajectories so that the policy improvement learned from imaginary
trajectories can be easily generalized to real trajectories. We demonstrate
that our approach improves sample efficiency of model-based planning, and
achieves state-of-the-art performance on challenging visual control benchmarks.
