  This paper proposes a multi-level feature learning framework for human action
recognition using a single body-worn inertial sensor. The framework consists of
three phases, respectively designed to analyze signal-based (low-level),
components (mid-level) and semantic (high-level) information. Low-level
features capture the time and frequency domain property while mid-level
representations learn the composition of the action. The Max-margin Latent
Pattern Learning (MLPL) method is proposed to learn high-level semantic
descriptions of latent action patterns as the output of our framework. The
proposed method achieves the state-of-the-art performances, 88.7%, 98.8% and
72.6% (weighted F1 score) respectively, on Skoda, WISDM and OPP datasets.
