  Human activities within smart infrastructures generate a vast amount of IMU
data from the wearables worn by individuals. Many existing studies rely on such
sensory data for human activity recognition (HAR); however, one of the major
bottlenecks is their reliance on pre-annotated or labeled data. Manual
human-driven annotations are neither scalable nor efficient, whereas existing
auto-annotation techniques heavily depend on video signatures. Still,
video-based auto-annotation needs high computation resources and has privacy
concerns when the data from a personal space, like a smart-home, is transferred
to the cloud. This paper exploits the acoustic signatures generated from human
activities to label the wearables' IMU data at the edge, thus mitigating
resource requirement and data privacy concerns. We utilize acoustic-based
pre-trained HAR models for cross-modal labeling of the IMU data even when two
individuals perform simultaneous but different activities under the same
environmental context. We observe that non-overlapping acoustic gaps exist with
a high probability during the simultaneous activities performed by two
individuals in the environment's acoustic context, which helps us resolve the
overlapping activity signatures to label them individually. A principled
evaluation of the proposed approach on two real-life in-house datasets further
augmented to create a dual occupant setup, shows that the framework can
correctly annotate a significant volume of unlabeled IMU data from both
individuals with an accuracy of $\mathbf{82.59\%}$ ($\mathbf{\pm 17.94\%}$) and
$\mathbf{98.32\%}$ ($\mathbf{\pm 3.68\%}$), respectively, for a workshop and a
kitchen environment.
