  A motion taxonomy can encode manipulations as a binary-encoded
representation, which we refer to as motion codes. These motion codes innately
represent a manipulation action in an embedded space that describes the
motion's mechanical features, including contact and trajectory type. The key
advantage of using motion codes for embedding is that motions can be more
appropriately defined with robotic-relevant features, and their distances can
be more reasonably measured using these motion features. In this paper, we
develop a deep learning pipeline to extract motion codes from demonstration
videos in an unsupervised manner so that knowledge from these videos can be
properly represented and used for robots. Our evaluations show that motion
codes can be extracted from demonstrations of action in the EPIC-KITCHENS
dataset.
