  We present an autoencoder-based semi-supervised approach to classify
perceived human emotions from walking styles obtained from videos or
motion-captured data and represented as sequences of 3D poses. Given the motion
on each joint in the pose at each time step extracted from 3D pose sequences,
we hierarchically pool these joint motions in a bottom-up manner in the
encoder, following the kinematic chains in the human body. We also constrain
the latent embeddings of the encoder to contain the space of
psychologically-motivated affective features underlying the gaits. We train the
decoder to reconstruct the motions per joint per time step in a top-down manner
from the latent embeddings. For the annotated data, we also train a classifier
to map the latent embeddings to emotion labels. Our semi-supervised approach
achieves a mean average precision of 0.84 on the Emotion-Gait benchmark
dataset, which contains both labeled and unlabeled gaits collected from
multiple sources. We outperform current state-of-art algorithms for both
emotion recognition and action recognition from 3D gaits by 7%--23% on the
absolute. More importantly, we improve the average precision by 10%--50% on the
absolute on classes that each makes up less than 25% of the labeled part of the
Emotion-Gait benchmark dataset.
