  Natural language is perhaps the most flexible and intuitive way for humans to
communicate tasks to a robot. Prior work in imitation learning typically
requires each task be specified with a task id or goal image -- something that
is often impractical in open-world environments. On the other hand, previous
approaches in instruction following allow agent behavior to be guided by
language, but typically assume structure in the observations, actuators, or
language that limit their applicability to complex settings like robotics. In
this work, we present a method for incorporating free-form natural language
conditioning into imitation learning. Our approach learns perception from
pixels, natural language understanding, and multitask continuous control
end-to-end as a single neural network. Unlike prior work in imitation learning,
our method is able to incorporate unlabeled and unstructured demonstration data
(i.e. no task or language labels). We show this dramatically improves language
conditioned performance, while reducing the cost of language annotation to less
than 1% of total data. At test time, a single language conditioned visuomotor
policy trained with our method can perform a wide variety of robotic
manipulation skills in a 3D environment, specified only with natural language
descriptions of each task (e.g. "open the drawer...now pick up the block...now
press the green button..."). To scale up the number of instructions an agent
can follow, we propose combining text conditioned policies with large
pretrained neural language models. We find this allows a policy to be robust to
many out-of-distribution synonym instructions, without requiring new
demonstrations. See videos of a human typing live text commands to our agent at
language-play.github.io
