  Non-parametric episodic memory can be used to quickly latch onto
high-rewarded experience in reinforcement learning tasks. In contrast to
parametric deep reinforcement learning approaches in which reward signals need
to be back-propagated slowly, these methods only need to discover the solution
once, and may then repeatedly solve the task. However, episodic control
solutions are stored in discrete tables, and this approach has so far only been
applied to discrete action space problems. Therefore, this paper introduces
Continuous Episodic Control (CEC), a novel non-parametric episodic memory
algorithm for sequential decision making in problems with a continuous action
space. Results on several sparse-reward continuous control environments show
that our proposed method learns faster than state-of-the-art model-free RL and
memory-augmented RL algorithms, while maintaining good long-run performance as
well. In short, CEC can be a fast approach for learning in continuous control
tasks.
