  Various types of sensors can be used for Human Activity Recognition (HAR),
and each of them has different strengths and weaknesses. Sometimes a single
sensor cannot fully observe the user's motions from its perspective, which
causes wrong predictions. While sensor fusion provides more information for
HAR, it comes with many inherent drawbacks like user privacy and acceptance,
costly set-up, operation, and maintenance. To deal with this problem, we
propose Virtual Fusion - a new method that takes advantage of unlabeled data
from multiple time-synchronized sensors during training, but only needs one
sensor for inference. Contrastive learning is adopted to exploit the
correlation among sensors. Virtual Fusion gives significantly better accuracy
than training with the same single sensor, and in some cases, it even surpasses
actual fusion using multiple sensors at test time. We also extend this method
to a more general version called Actual Fusion within Virtual Fusion (AFVF),
which uses a subset of training sensors during inference. Our method achieves
state-of-the-art accuracy and F1-score on UCI-HAR and PAMAP2 benchmark
datasets. Implementation is available upon request.
