  Recent advances in remote health monitoring systems have significantly
benefited patients and played a crucial role in improving their quality of
life. However, while physiological health-focused solutions have demonstrated
increasing success and maturity, mental health-focused applications have seen
comparatively limited success in spite of the fact that stress and anxiety
disorders are among the most common issues people deal with in their daily
lives. In the hopes of furthering progress in this domain through the
development of a more robust analytic framework for the measurement of
indicators of mental health, we propose a multi-modal semi-supervised framework
for tracking physiological precursors of the stress response. Our methodology
enables utilizing multi-modal data of differing domains and resolutions from
wearable devices and leveraging them to map short-term episodes to semantically
efficient embeddings for a given task. Additionally, we leverage an
inter-modality contrastive objective, with the advantages of rendering our
framework both modular and scalable. The focus on optimizing both local and
global aspects of our embeddings via a hierarchical structure renders
transferring knowledge and compatibility with other devices easier to achieve.
In our pipeline, a task-specific pooling based on an attention mechanism, which
estimates the contribution of each modality on an instance level, computes the
final embeddings for observations. This additionally provides a thorough
diagnostic insight into the data characteristics and highlights the importance
of signals in the broader view of predicting episodes annotated per mental
health status. We perform training experiments using a corpus of real-world
data on perceived stress, and our results demonstrate the efficacy of the
proposed approach in performance improvements.
