  Many Reinforcement Learning (RL) approaches use joint control signals
(positions, velocities, torques) as action space for continuous control tasks.
We propose to lift the action space to a higher level in the form of subgoals
for a motion generator (a combination of motion planner and trajectory
executor). We argue that, by lifting the action space and by leveraging
sampling-based motion planners, we can efficiently use RL to solve complex,
long-horizon tasks that could not be solved with existing RL methods in the
original action space. We propose ReLMoGen -- a framework that combines a
learned policy to predict subgoals and a motion generator to plan and execute
the motion needed to reach these subgoals. To validate our method, we apply
ReLMoGen to two types of tasks: 1) Interactive Navigation tasks, navigation
problems where interactions with the environment are required to reach the
destination, and 2) Mobile Manipulation tasks, manipulation tasks that require
moving the robot base. These problems are challenging because they are usually
long-horizon, hard to explore during training, and comprise alternating phases
of navigation and interaction. Our method is benchmarked on a diverse set of
seven robotics tasks in photo-realistic simulation environments. In all
settings, ReLMoGen outperforms state-of-the-art Reinforcement Learning and
Hierarchical Reinforcement Learning baselines. ReLMoGen also shows outstanding
transferability between different motion generators at test time, indicating a
great potential to transfer to real robots.
