  Many continuous control tasks have easily formulated objectives, yet using
them directly as a reward in reinforcement learning (RL) leads to suboptimal
policies. Therefore, many classical control tasks guide RL training using
complex rewards, which require tedious hand-tuning. We automate the reward
search with AutoRL, an evolutionary layer over standard RL that treats reward
tuning as hyperparameter optimization and trains a population of RL agents to
find a reward that maximizes the task objective. AutoRL, evaluated on four
Mujoco continuous control tasks over two RL algorithms, shows improvements over
baselines, with the the biggest uplift for more complex tasks. The video can be
found at: \url{https://youtu.be/svdaOFfQyC8}.
