  In continuous action domains, standard deep reinforcement learning algorithms
like DDPG suffer from inefficient exploration when facing sparse or deceptive
reward problems. Conversely, evolutionary and developmental methods focusing on
exploration like Novelty Search, Quality-Diversity or Goal Exploration
Processes explore more robustly but are less efficient at fine-tuning policies
using gradient descent. In this paper, we present the GEP-PG approach, taking
the best of both worlds by sequentially combining a Goal Exploration Process
and two variants of DDPG. We study the learning performance of these components
and their combination on a low dimensional deceptive reward problem and on the
larger Half-Cheetah benchmark. We show that DDPG fails on the former and that
GEP-PG improves over the best DDPG variant in both environments. Supplementary
videos and discussion can be found at http://frama.link/gep_pg, the code at
http://github.com/flowersteam/geppg.
