  Research on Inverse Reinforcement Learning (IRL) from third-person videos has
shown encouraging results on removing the need for manual reward design for
robotic tasks. However, most prior works are still limited by training from a
relatively restricted domain of videos. In this paper, we argue that the true
potential of third-person IRL lies in increasing the diversity of videos for
better scaling. To learn a reward function from diverse videos, we propose to
perform graph abstraction on the videos followed by temporal matching in the
graph space to measure the task progress. Our insight is that a task can be
described by entity interactions that form a graph, and this graph abstraction
can help remove irrelevant information such as textures, resulting in more
robust reward functions. We evaluate our approach, GraphIRL, on
cross-embodiment learning in X-MAGICAL and learning from human demonstrations
for real-robot manipulation. We show significant improvements in robustness to
diverse video demonstrations over previous approaches, and even achieve better
results than manual reward design on a real robot pushing task. Videos are
available at https://sateeshkumar21.github.io/GraphIRL .
