  Learning discriminative representation from the complex spatio-temporal
dynamic space is essential for video recognition. On top of those stylized
spatio-temporal computational units, further refining the learnt feature with
axial contexts is demonstrated to be promising in achieving this goal. However,
previous works generally focus on utilizing a single kind of contexts to
calibrate entire feature channels and could hardly apply to deal with diverse
video activities. The problem can be tackled by using pair-wise spatio-temporal
attentions to recompute feature response with cross-axis contexts at the
expense of heavy computations. In this paper, we propose an efficient feature
refinement method that decomposes the feature channels into several groups and
separately refines them with different axial contexts in parallel. We refer
this lightweight feature calibration as group contextualization (GC).
Specifically, we design a family of efficient element-wise calibrators, i.e.,
ECal-G/S/T/L, where their axial contexts are information dynamics aggregated
from other axes either globally or locally, to contextualize feature channel
groups. The GC module can be densely plugged into each residual layer of the
off-the-shelf video networks. With little computational overhead, consistent
improvement is observed when plugging in GC on different networks. By utilizing
calibrators to embed feature with four different kinds of contexts in parallel,
the learnt representation is expected to be more resilient to diverse types of
activities. On videos with rich temporal variations, empirically GC can boost
the performance of 2D-CNN (e.g., TSN and TSM) to a level comparable to the
state-of-the-art video networks. Code is available at
https://github.com/haoyanbin918/Group-Contextualization.
