  Hierarchical Imitation Learning (HIL) is an effective way for robots to learn
sub-skills from long-horizon unsegmented demonstrations. However, the learned
hierarchical structure lacks the mechanism to transfer across multi-tasks or to
new tasks, which makes them have to learn from scratch when facing a new
situation. Transferring and reorganizing modular sub-skills require fast
adaptation ability of the whole hierarchical structure. In this work, we
propose Dual Meta Imitation Learning (DMIL), a hierarchical meta imitation
learning method where the high-level network and sub-skills are iteratively
meta-learned with model-agnostic meta-learning. DMIL uses the likelihood of
state-action pairs from each sub-skill as the supervision for the high-level
network adaptation, and use the adapted high-level network to determine
different data set for each sub-skill adaptation. We theoretically prove the
convergence of the iterative training process of DMIL and establish the
connection between DMIL and Expectation-Maximization algorithm. Empirically, we
achieve state-of-the-art few-shot imitation learning performance on the
Meta-world \cite{metaworld} benchmark and competitive results on long-horizon
tasks of Kitchen environments.
