  How to model fine-grained spatial-temporal dynamics in videos has been a
challenging problem for action recognition. It requires learning deep and rich
features with superior distinctiveness for the subtle and abstract motions.
Most existing methods generate features of a layer in a pure feedforward
manner, where the information moves in one direction from inputs to outputs.
And they rely on stacking more layers to obtain more powerful features,
bringing extra non-negligible overheads. In this paper, we propose an Adaptive
Recursive Circle (ARC) framework, a fine-grained decorator for pure feedforward
layers. It inherits the operators and parameters of the original layer but is
slightly different in the use of those operators and parameters. Specifically,
the input of the layer is treated as an evolving state, and its update is
alternated with the feature generation. At each recursive step, the input state
is enriched by the previously generated features and the feature generation is
made with the newly updated input state. We hope the ARC framework can
facilitate fine-grained action recognition by introducing deeply refined
features and multi-scale receptive fields at a low cost. Significant
improvements over feedforward baselines are observed on several benchmarks. For
example, an ARC-equipped TSM-ResNet18 outperforms TSM-ResNet50 with 48% fewer
FLOPs and 52% model parameters on Something-Something V1 and Diving48.
