  We present a deep imitation learning framework for robotic bimanual
manipulation in a continuous state-action space. A core challenge is to
generalize the manipulation skills to objects in different locations. We
hypothesize that modeling the relational information in the environment can
significantly improve generalization. To achieve this, we propose to (i)
decompose the multi-modal dynamics into elemental movement primitives, (ii)
parameterize each primitive using a recurrent graph neural network to capture
interactions, and (iii) integrate a high-level planner that composes primitives
sequentially and a low-level controller to combine primitive dynamics and
inverse kinematics control. Our model is a deep, hierarchical, modular
architecture. Compared to baselines, our model generalizes better and achieves
higher success rates on several simulated bimanual robotic manipulation tasks.
We open source the code for simulation, data, and models at:
https://github.com/Rose-STL-Lab/HDR-IL.
