  The process of learning a manipulation task depends strongly on the action
space used for exploration: posed in the incorrect action space, solving a task
with reinforcement learning can be drastically inefficient. Additionally,
similar tasks or instances of the same task family impose latent manifold
constraints on the most effective action space: the task family can be best
solved with actions in a manifold of the entire action space of the robot.
Combining these insights we present LASER, a method to learn latent action
spaces for efficient reinforcement learning. LASER factorizes the learning
problem into two sub-problems, namely action space learning and policy learning
in the new action space. It leverages data from similar manipulation task
instances, either from an offline expert or online during policy learning, and
learns from these trajectories a mapping from the original to a latent action
space. LASER is trained as a variational encoder-decoder model to map raw
actions into a disentangled latent action space while maintaining action
reconstruction and latent space dynamic consistency. We evaluate LASER on two
contact-rich robotic tasks in simulation, and analyze the benefit of policy
learning in the generated latent action space. We show improved sample
efficiency compared to the original action space from better alignment of the
action space to the task space, as we observe with visualizations of the
learned action space manifold. Additional details:
https://www.pair.toronto.edu/laser
