  Imitation learning from human demonstrations can teach robots complex
manipulation skills, but is time-consuming and labor intensive. In contrast,
Task and Motion Planning (TAMP) systems are automated and excel at solving
long-horizon tasks, but they are difficult to apply to contact-rich tasks. In
this paper, we present Human-in-the-Loop Task and Motion Planning (HITL-TAMP),
a novel system that leverages the benefits of both approaches. The system
employs a TAMP-gated control mechanism, which selectively gives and takes
control to and from a human teleoperator. This enables the human teleoperator
to manage a fleet of robots, maximizing data collection efficiency. The
collected human data is then combined with an imitation learning framework to
train a TAMP-gated policy, leading to superior performance compared to training
on full task demonstrations. We compared HITL-TAMP to a conventional
teleoperation system -- users gathered more than 3x the number of demos given
the same time budget. Furthermore, proficient agents (75\%+ success) could be
trained from just 10 minutes of non-expert teleoperation data. Finally, we
collected 2.1K demos with HITL-TAMP across 12 contact-rich, long-horizon tasks
and show that the system often produces near-perfect agents. Videos and
additional results at https://hitltamp.github.io .
