  Reinforcement learning is a promising approach for learning control policies
for robot tasks. However, specifying complex tasks (e.g., with multiple
objectives and safety constraints) can be challenging, since the user must
design a reward function that encodes the entire task. Furthermore, the user
often needs to manually shape the reward to ensure convergence of the learning
algorithm. We propose a language for specifying complex control tasks, along
with an algorithm that compiles specifications in our language into a reward
function and automatically performs reward shaping. We implement our approach
in a tool called SPECTRL, and show that it outperforms several state-of-the-art
baselines.
