  Based on the progress of image recognition, video recognition has been
extensively studied recently. However, most of the existing methods are focused
on short-term but not long-term video recognition, called contextual video
recognition. To address contextual video recognition, we use convolutional
recurrent neural networks (ConvRNNs) having a rich spatio-temporal information
processing capability, but ConvRNNs requires extensive computation that slows
down training. In this paper, inspired by the normalization and detrending
methods, we propose adaptive detrending (AD) for temporal normalization in
order to accelerate the training of ConvRNNs, especially for convolutional
gated recurrent unit (ConvGRU). AD removes internal covariate shift within a
sequence of each neuron in recurrent neural networks (RNNs) by subtracting a
trend. In the experiments for contextual recognition on ConvGRU, the results
show that (1) ConvGRU clearly outperforms the feed-forward neural networks, (2)
AD consistently offers a significant training acceleration and generalization
improvement, and (3) AD is further improved by collaborating with the existing
normalization methods.
