  We present an object-centric approach to empower robots to learn vision-based
manipulation skills from human videos. We investigate the problem of imitating
robot manipulation from a single human video in the open-world setting, where a
robot must learn to manipulate novel objects from one video demonstration. We
introduce ORION, an algorithm that tackles the problem by extracting an
object-centric manipulation plan from a single RGB-D video and deriving a
policy that conditions on the extracted plan. Our method enables the robot to
learn from videos captured by daily mobile devices such as an iPad and
generalize the policies to deployment environments with varying visual
backgrounds, camera angles, spatial layouts, and novel object instances. We
systematically evaluate our method on both short-horizon and long-horizon
tasks, demonstrating the efficacy of ORION in learning from a single human
video in the open world. Videos can be found in the project website
https://ut-austin-rpl.github.io/ORION-release.
