  It is inevitably crucial to classify sleep stage for the diagnosis of various
diseases. However, existing automated diagnosis methods mostly adopt the
"gold-standard" lectroencephalogram (EEG) or other uni-modal sensing signal of
the PolySomnoGraphy (PSG) machine in hospital, that are expensive, importable
and therefore unsuitable for point-of-care monitoring at home. To enable the
sleep stage monitoring at home, in this paper, we analyze the relationship
between infrared videos and the EEG signal and propose a new task: to classify
the sleep stage using infrared videos by distilling useful knowledge from EEG
signals to the visual ones. To establish a solid cross-modal benchmark for this
application, we develop a new dataset termed as Seeing your Sleep Stage via
Infrared Video and EEG ($S^3VE$). $S^3VE$ is a large-scale dataset including
synchronized infrared video and EEG signal for sleep stage classification,
including 105 subjects and 154,573 video clips that is more than 1100 hours
long. Our contributions are not limited to datasets but also about a novel
cross-modal distillation baseline model namely the structure-aware contrastive
distillation (SACD) to distill the EEG knowledge to infrared video features.
The SACD achieved the state-of-the-art performances on both our $S^3VE$ and the
existing cross-modal distillation benchmark. Both the benchmark and the
baseline methods will be released to the community. We expect to raise more
attentions and promote more developments in the sleep stage classification and
more importantly the cross-modal distillation from clinical signal/media to the
conventional media.
