  Existing imitation learning works mainly assume that the demonstrator who
collects demonstrations shares the same dynamics as the imitator. However, the
assumption limits the usage of imitation learning, especially when collecting
demonstrations for the imitator is difficult. In this paper, we study
out-of-dynamics imitation learning (OOD-IL), which relaxes the assumption to
that the demonstrator and the imitator have the same state spaces but could
have different action spaces and dynamics. OOD-IL enables imitation learning to
utilize demonstrations from a wide range of demonstrators but introduces a new
challenge: some demonstrations cannot be achieved by the imitator due to the
different dynamics. Prior works try to filter out such demonstrations by
feasibility measurements, but ignore the fact that the demonstrations exhibit a
multimodal distribution since the different demonstrators may take different
policies in different dynamics. We develop a better transferability measurement
to tackle this newly-emerged challenge. We firstly design a novel
sequence-based contrastive clustering algorithm to cluster demonstrations from
the same mode to avoid the mutual interference of demonstrations from different
modes, and then learn the transferability of each demonstration with an
adversarial-learning based algorithm in each cluster. Experiment results on
several MuJoCo environments, a driving environment, and a simulated robot
environment show that the proposed transferability measurement more accurately
finds and down-weights non-transferable demonstrations and outperforms prior
works on the final imitation learning performance. We show the videos of our
experiment results on our website.
