  We consider an active visual exploration scenario, where an agent must
intelligently select its camera motions to efficiently reconstruct the full
environment from only a limited set of narrow field-of-view glimpses. While the
agent has full observability of the environment during training, it has only
partial observability once deployed, being constrained by what portions it has
seen and what camera motions are permissible. We introduce sidekick policy
learning to capitalize on this imbalance of observability. The main idea is a
preparatory learning phase that attempts simplified versions of the eventual
exploration task, then guides the agent via reward shaping or initial policy
supervision. To support interpretation of the resulting policies, we also
develop a novel policy visualization technique. Results on active visual
exploration tasks with 360 scenes and 3D objects show that sidekicks
consistently improve performance and convergence rates over existing methods.
Code, data and demos are available.
