  Exploration of indoor environments has recently experienced a significant
interest, also thanks to the introduction of deep neural agents built in a
hierarchical fashion and trained with Deep Reinforcement Learning (DRL) on
simulated environments. Current state-of-the-art methods employ a dense
extrinsic reward that requires the complete a priori knowledge of the layout of
the training environment to learn an effective exploration policy. However,
such information is expensive to gather in terms of time and resources. In this
work, we propose to train the model with a purely intrinsic reward signal to
guide exploration, which is based on the impact of the robot's actions on its
internal representation of the environment. So far, impact-based rewards have
been employed for simple tasks and in procedurally generated synthetic
environments with countable states. Since the number of states observable by
the agent in realistic indoor environments is non-countable, we include a
neural-based density model and replace the traditional count-based
regularization with an estimated pseudo-count of previously visited states. The
proposed exploration approach outperforms DRL-based competitors relying on
intrinsic rewards and surpasses the agents trained with a dense extrinsic
reward computed with the environment layouts. We also show that a robot
equipped with the proposed approach seamlessly adapts to point-goal navigation
and real-world deployment.
