  Modern sleep monitoring development is shifting towards the use of
unobtrusive sensors combined with algorithms for automatic sleep scoring. Many
different combinations of wet and dry electrodes, ear-centered,
forehead-mounted or headband-inspired designs have been proposed, alongside an
ever growing variety of machine learning algorithms for automatic sleep
scoring. In this paper, we compare 13 different, realistic sensor setups
derived from the same data set and analysed with the same pipeline. We find
that all setups which include both a lateral and an EOG derivation show
similar, state-of-the-art performance, with average Cohen's kappa values of at
least 0.80. This indicates that electrode distance, rather than position, is
important for accurate sleep scoring. Finally, based on the results presented,
we argue that with the current competitive performance of automated staging
approaches, there is an urgent need for establishing an improved benchmark
beyond current single human rater scoring.
