  In Time Series Classification (TSC), temporal pooling methods that consider
sequential information have been proposed. However, we found that each temporal
pooling has a distinct mechanism, and can perform better or worse depending on
time series data. We term this fixed pooling mechanism a single perspective of
temporal poolings. In this paper, we propose a novel temporal pooling method
with diverse perspective learning: Selection over Multiple Temporal Poolings
(SoM-TP). SoM-TP dynamically selects the optimal temporal pooling among
multiple methods for each data by attention. The dynamic pooling selection is
motivated by the ensemble concept of Multiple Choice Learning (MCL), which
selects the best among multiple outputs. The pooling selection by SoM-TP's
attention enables a non-iterative pooling ensemble within a single classifier.
Additionally, we define a perspective loss and Diverse Perspective Learning
Network (DPLN). The loss works as a regularizer to reflect all the pooling
perspectives from DPLN. Our perspective analysis using Layer-wise Relevance
Propagation (LRP) reveals the limitation of a single perspective and ultimately
demonstrates diverse perspective learning of SoM-TP. We also show that SoM-TP
outperforms CNN models based on other temporal poolings and state-of-the-art
models in TSC with extensive UCR/UEA repositories.
