  The growing number of action classes has posed a new challenge for video
understanding, making Zero-Shot Action Recognition (ZSAR) a thriving direction.
The ZSAR task aims to recognize target (unseen) actions without training
examples by leveraging semantic representations to bridge seen and unseen
actions. However, due to the complexity and diversity of actions, it remains
challenging to semantically represent action classes and transfer knowledge
from seen data. In this work, we propose an ER-enhanced ZSAR model inspired by
an effective human memory technique Elaborative Rehearsal (ER), which involves
elaborating a new concept and relating it to known concepts. Specifically, we
expand each action class as an Elaborative Description (ED) sentence, which is
more discriminative than a class name and less costly than manual-defined
attributes. Besides directly aligning class semantics with videos, we
incorporate objects from the video as Elaborative Concepts (EC) to improve
video semantics and generalization from seen actions to unseen actions. Our
ER-enhanced ZSAR model achieves state-of-the-art results on three existing
benchmarks. Moreover, we propose a new ZSAR evaluation protocol on the Kinetics
dataset to overcome limitations of current benchmarks and demonstrate the first
case where ZSAR performance is comparable to few-shot learning baselines on
this more realistic setting. We will release our codes and collected EDs at
https://github.com/DeLightCMU/ElaborativeRehearsal.
