  Reinforcement learning for embodied agents is a challenging problem. The
accumulated reward to be optimized is often a very rugged function, and
gradient methods are impaired by many local optimizers. We demonstrate, in an
experimental setting, that incorporating an intrinsic reward can smoothen the
optimization landscape while preserving the global optimizers of interest. We
show that policy gradient optimization for locomotion in a complex morphology
is significantly improved when supplementing the extrinsic reward by an
intrinsic reward defined in terms of the mutual information of time consecutive
sensor readings.
