  It is expensive and time-consuming to collect sufficient labeled data for
human activity recognition (HAR). Domain adaptation is a promising approach for
cross-domain activity recognition. Existing methods mainly focus on adapting
cross-domain representations via domain-level, class-level, or sample-level
distribution matching. However, they might fail to capture the fine-grained
locality information in activity data. The domain- and class-level matching are
too coarse that may result in under-adaptation, while sample-level matching may
be affected by the noise seriously and eventually cause over-adaptation. In
this paper, we propose substructure-level matching for domain adaptation (SSDA)
to better utilize the locality information of activity data for accurate and
efficient knowledge transfer. Based on SSDA, we propose an optimal
transport-based implementation, Substructural Optimal Transport (SOT), for
cross-domain HAR. We obtain the substructures of activities via clustering
methods and seeks the coupling of the weighted substructures between different
domains. We conduct comprehensive experiments on four public activity
recognition datasets (i.e. UCI-DSADS, UCI-HAR, USC-HAD, PAMAP2), which
demonstrates that SOT significantly outperforms other state-of-the-art methods
w.r.t classification accuracy (9%+ improvement). In addition, our mehtod is 5x
faster than traditional OT-based DA methods with the same hyper-parameters.
