  In this work, we investigate Active Vision Reinforcement Learning
(ActiveVision-RL), where an embodied agent simultaneously learns action policy
for the task while also controlling its visual observations in partially
observable environments. We denote the former as motor policy and the latter as
sensory policy. For example, humans solve real world tasks by hand manipulation
(motor policy) together with eye movements (sensory policy). ActiveVision-RL
poses challenges on coordinating two policies given their mutual influence. We
propose SUGARL, Sensorimotor Understanding Guided Active Reinforcement
Learning, a framework that models motor and sensory policies separately, but
jointly learns them using with an intrinsic sensorimotor reward. This learnable
reward is assigned by sensorimotor reward module, incentivizes the sensory
policy to select observations that are optimal to infer its own motor action,
inspired by the sensorimotor stage of humans. Through a series of experiments,
we show the effectiveness of our method across a range of observability
conditions and its adaptability to existed RL algorithms. The sensory policies
learned through our method are observed to exhibit effective active vision
strategies.
