  In many real-world imitation learning tasks, the demonstrator and the learner
have to act under different observation spaces. This situation brings
significant obstacles to existing imitation learning approaches, since most of
them learn policies under homogeneous observation spaces. On the other hand,
previous studies under different observation spaces have strong assumptions
that these two observation spaces coexist during the entire learning process.
However, in reality, the observation coexistence will be limited due to the
high cost of acquiring expert observations. In this work, we study this
challenging problem with limited observation coexistence under heterogeneous
observations: Heterogeneously Observable Imitation Learning (HOIL). We identify
two underlying issues in HOIL: the dynamics mismatch and the support mismatch,
and further propose the Importance Weighting with REjection (IWRE) algorithm
based on importance weighting and learning with rejection to solve HOIL
problems. Experimental results show that IWRE can solve various HOIL tasks,
including the challenging tasks of transforming the vision-based demonstrations
to random access memory (RAM)-based policies in the Atari domain, even with
limited visual observations.
