  Attention mechanisms are widely used to dramatically improve deep learning
model performance in various fields. However, their general ability to improve
the performance of physiological signal deep learning model is immature. In
this study, we experimentally analyze four attention mechanisms (e.g.,
squeeze-and-excitation, non-local, convolutional block attention module, and
multi-head self-attention) and three convolutional neural network (CNN)
architectures (e.g., VGG, ResNet, and Inception) for two representative
physiological signal prediction tasks: the classification for predicting
hypotension and the regression for predicting cardiac output (CO). We evaluated
multiple combinations for performance and convergence of physiological signal
deep learning model. Accordingly, the CNN models with the spatial attention
mechanism showed the best performance in the classification problem, whereas
the channel attention mechanism achieved the lowest error in the regression
problem. Moreover, the performance and convergence of the CNN models with
attention mechanisms were better than stand-alone self-attention models in both
problems. Hence, we verified that convolutional operation and attention
mechanisms are complementary and provide faster convergence time, despite the
stand-alone self-attention models requiring fewer parameters.
