  Research in human action recognition has accelerated significantly since the
introduction of powerful machine learning tools such as Convolutional Neural
Networks (CNNs). However, effective and efficient methods for incorporation of
temporal information into CNNs are still being actively explored in the recent
literature. Motivated by the popular recurrent attention models in the research
area of natural language processing, we propose the Attention-based Temporal
Weighted CNN (ATW), which embeds a visual attention model into a temporal
weighted multi-stream CNN. This attention model is simply implemented as
temporal weighting yet it effectively boosts the recognition performance of
video representations. Besides, each stream in the proposed ATW framework is
capable of end-to-end training, with both network parameters and temporal
weights optimized by stochastic gradient descent (SGD) with backpropagation.
Our experiments show that the proposed attention mechanism contributes
substantially to the performance gains with the more discriminative snippets by
focusing on more relevant video segments.
