  The integration of Reinforcement Learning (RL) and Evolutionary Algorithms
(EAs) aims at simultaneously exploiting the sample efficiency as well as the
diversity and robustness of the two paradigms. Recently, hybrid learning
frameworks based on this principle have achieved great success in various
challenging robot control tasks. However, in these methods, policies from the
genetic population are evaluated via interactions with the real environments,
limiting their applicability in computationally expensive problems. In this
work, we propose Surrogate-assisted Controller (SC), a novel and efficient
module that can be integrated into existing frameworks to alleviate the
computational burden of EAs by partially replacing the expensive policy
evaluation. The key challenge in applying this module is to prevent the
optimization process from being misled by the possible false minima introduced
by the surrogate. To address this issue, we present two strategies for SC to
control the workflow of hybrid frameworks. Experiments on six continuous
control tasks from the OpenAI Gym platform show that SC can not only
significantly reduce the cost of fitness evaluations, but also boost the
performance of the original hybrid frameworks with collaborative learning and
evolutionary processes.
