Addressing Sample Complexity in Visual Tasks Using HER and Hallucinatory
  GANs
  Reinforcement Learning (RL) algorithms typically require millions of
environment interactions to learn successful policies in sparse reward
settings. Hindsight Experience Replay (HER) was introduced as a technique to
increase sample efficiency by reimagining unsuccessful trajectories as
successful ones by altering the originally intended goals. However, it cannot
be directly applied to visual environments where goal states are often
characterized by the presence of distinct visual features. In this work, we
show how visual trajectories can be hallucinated to appear successful by
altering agent observations using a generative model trained on relatively few
snapshots of the goal. We then use this model in combination with HER to train
RL agents in visual settings. We validate our approach on 3D navigation tasks
and a simulated robotics application and show marked improvement over baselines
derived from previous work.
