  Background: Despite the tremendous progress recently made towards automatic
sleep staging in adults, it is currently unknown if the most advanced
algorithms generalize to the pediatric population, which displays distinctive
characteristics in overnight polysomnography (PSG). Methods: To answer the
question, in this work, we conduct a large-scale comparative study on the
state-of-the-art deep learning methods for pediatric automatic sleep staging.
Six different deep neural networks with diverging features are adopted to
evaluate a sample of more than 1,200 children across a wide spectrum of
obstructive sleep apnea (OSA) severity. Results: Our experimental results show
that the individual performance of automated pediatric sleep stagers when
evaluated on new subjects is equivalent to the expert-level one reported on
adults. Combining the six stagers into ensemble models further boosts the
staging accuracy, reaching an overall accuracy of 88.8%, a Cohen's kappa of
0.852, and a macro F1-score of 85.8%. At the same time, the ensemble models
lead to reduced predictive uncertainty. The results also show that the studied
algorithms and their ensembles are robust to concept drift when the training
and test data were recorded seven months apart and after clinical intervention.
Conclusion: However, we show that the improvements in the staging performance
are not necessarily clinically significant although the ensemble models lead to
more favorable clinical measures than the six standalone models. Significance:
Detailed analyses further demonstrate "almost perfect" agreement between the
automatic stagers to one another and their similar patterns on the staging
errors, suggesting little room for improvement.
