  We aim to control a robot to physically behave in the real world following
any high-level language command like "cartwheel" or "kick". Although human
motion datasets exist, this task remains particularly challenging since
generative models can produce physically unrealistic motions, which will be
more severe for robots due to different body structures and physical
properties. Deploying such a motion to a physical robot can cause even greater
difficulties due to the sim2real gap. We develop LAnguage-Guided mOtion cONtrol
(LAGOON), a multi-phase reinforcement learning (RL) method to generate
physically realistic robot motions under language commands. LAGOON first
leverages a pretrained model to generate a human motion from a language
command. Then an RL phase trains a control policy in simulation to mimic the
generated human motion. Finally, with domain randomization, our learned policy
can be deployed to a quadrupedal robot, leading to a quadrupedal robot that can
take diverse behaviors in the real world under natural language commands
