  While recent progress in deep reinforcement learning has enabled robots to
learn complex behaviors, tasks with long horizons and sparse rewards remain an
ongoing challenge. In this work, we propose an effective reward shaping method
through predictive coding to tackle sparse reward problems. By learning
predictive representations offline and using these representations for reward
shaping, we gain access to reward signals that understand the structure and
dynamics of the environment. In particular, our method achieves better learning
by providing reward signals that 1) understand environment dynamics 2)
emphasize on features most useful for learning 3) resist noise in learned
representations through reward accumulation. We demonstrate the usefulness of
this approach in different domains ranging from robotic manipulation to
navigation, and we show that reward signals produced through predictive coding
are as effective for learning as hand-crafted rewards.
