  Sleep is a complex physiological process evaluated through various modalities
recording electrical brain, cardiac, and respiratory activities. We curate a
large polysomnography dataset from over 14,000 participants comprising over
100,000 hours of multi-modal sleep recordings. Leveraging this extensive
dataset, we developed SleepFM, the first multi-modal foundation model for sleep
analysis. We show that a novel leave-one-out approach for contrastive learning
significantly improves downstream task performance compared to representations
from standard pairwise contrastive learning. A logistic regression model
trained on SleepFM's learned embeddings outperforms an end-to-end trained
convolutional neural network (CNN) on sleep stage classification (macro AUROC
0.88 vs 0.72 and macro AUPRC 0.72 vs 0.48) and sleep disordered breathing
detection (AUROC 0.85 vs 0.69 and AUPRC 0.77 vs 0.61). Notably, the learned
embeddings achieve 48% top-1 average accuracy in retrieving the corresponding
recording clips of other modalities from 90,000 candidates. This work
demonstrates the value of holistic multi-modal sleep modeling to fully capture
the richness of sleep recordings. SleepFM is open source and available at
https://github.com/rthapa84/sleepfm-codebase.
