  Feature extraction is crucial for human activity recognition (HAR) using
body-worn movement sensors. Recently, learned representations have been used
successfully, offering promising alternatives to manually engineered features.
Our work focuses on effective use of small amounts of labeled data and the
opportunistic exploitation of unlabeled data that are straightforward to
collect in mobile and ubiquitous computing scenarios. We hypothesize and
demonstrate that explicitly considering the temporality of sensor data at
representation level plays an important role for effective HAR in challenging
scenarios. We introduce the Contrastive Predictive Coding (CPC) framework to
human activity recognition, which captures the long-term temporal structure of
sensor data streams. Through a range of experimental evaluations on real-life
recognition tasks, we demonstrate its effectiveness for improved HAR. CPC-based
pre-training is self-supervised, and the resulting learned representations can
be integrated into standard activity chains. It leads to significantly improved
recognition performance when only small amounts of labeled training data are
available, thereby demonstrating the practical value of our approach.
