  Studies have shown evolution strategies (ES) to be a promising approach for
reinforcement learning (RL) with deep neural networks. However, the issue of
high sample complexity persists in applications of ES to deep RL over long
horizons. This paper is the first to address the shortcoming of today's methods
via a novel neuroevolutionary multitasking (NuEMT) algorithm, designed to
transfer information from a set of auxiliary tasks (of short episode length) to
the target (full length) RL task at hand. The auxiliary tasks, extracted from
the target, allow an agent to update and quickly evaluate policies on shorter
time horizons. The evolved skills are then transferred to guide the longer and
harder task towards an optimal policy. We demonstrate that the NuEMT algorithm
achieves data-efficient evolutionary RL, reducing expensive agent-environment
interaction data requirements. Our key algorithmic contribution in this setting
is to introduce, for the first time, a multitask skills transfer mechanism
based on the statistical importance sampling technique. In addition, an
adaptive resource allocation strategy is utilized to assign computational
resources to auxiliary tasks based on their gleaned usefulness. Experiments on
a range of continuous control tasks from the OpenAI Gym confirm that our
proposed algorithm is efficient compared to recent ES baselines.
