  Learning from demonstration (LfD) provides a fast, intuitive and efficient
framework to program robot skills, which has gained growing interest both in
research and industrial applications. Most complex manipulation tasks are
long-term and involve a set of skill primitives. Thus it is crucial to have a
reliable coordination scheme that selects the correct sequence of skill
primitive and the correct parameters for each skill, under various scenarios.
Instead of relying on a precise simulator, this work proposes a
human-in-the-loop coordination framework for LfD skills that: builds
parameterized skill models from kinesthetic demonstrations; constructs a
geometric task network (GTN) on-the-fly from human instructions; learns a
hierarchical control policy incrementally during execution. This framework can
reduce significantly the manual design efforts, while improving the
adaptability to new scenes. We show on a 7-DoF robotic manipulator that the
proposed approach can teach complex industrial tasks such as bin sorting and
assembly in less than 30 minutes.
