  Multi-view action recognition (MVAR) leverages complementary temporal
information from different views to improve the learning performance. Obtaining
informative view-specific representation plays an essential role in MVAR.
Attention has been widely adopted as an effective strategy for discovering
discriminative cues underlying temporal data. However, most existing MVAR
methods only utilize attention to extract representation for each view
individually, ignoring the potential to dig latent patterns based on
mutual-support information in attention space. To this end, we propose a
collaborative attention mechanism (CAM) for solving the MVAR problem in this
paper. The proposed CAM detects the attention differences among multi-view, and
adaptively integrates frame-level information to benefit each other.
Specifically, we extend the long short-term memory (LSTM) to a Mutual-Aid RNN
(MAR) to achieve the multi-view collaboration process. CAM takes advantages of
view-specific attention pattern to guide another view and discover potential
information which is hard to be explored by itself. It paves a novel way to
leverage attention information and enhances the multi-view representation
learning. Extensive experiments on four action datasets illustrate the proposed
CAM achieves better results for each view and also boosts multi-view
performance.
