  State-of-the-art automatic sleep staging methods have already demonstrated
comparable reliability and superior time efficiency to manual sleep staging.
However, fully automatic black-box solutions are difficult to adapt into
clinical workflow and the interaction between explainable automatic methods and
the work of sleep technologists remains underexplored and inadequately
conceptualized. Thus, we propose a human-in-the-loop concept for sleep
analysis, presenting an automatic sleep staging model (aSAGA), that performs
effectively with both clinical polysomnographic recordings and home sleep
studies. To validate the model, extensive testing was conducted, employing a
preclinical validation approach with three retrospective datasets; open-access,
clinical, and research-driven. Furthermore, we validate the utilization of
uncertainty mapping to identify ambiguous regions, conceptualized as gray
areas, in automatic sleep analysis that warrants manual re-evaluation. The
results demonstrate that the automatic sleep analysis achieved a comparable
level of agreement with manual analysis across different sleep recording types.
Moreover, validation of the gray area concept revealed its potential to enhance
sleep staging accuracy and identify areas in the recordings where sleep
technologists struggle to reach a consensus. In conclusion, this study
introduces and validates a concept from explainable artificial intelligence
into sleep medicine and provides the basis for integrating human-in-the-loop
automatic sleep staging into clinical workflows, aiming to reduce black-box
criticism and the burden associated with manual sleep staging.
