  Temporal action localization (TAL) is an important and challenging problem in
video understanding. However, most existing TAL benchmarks are built upon the
coarse granularity of action classes, which exhibits two major limitations in
this task. First, coarse-level actions can make the localization models overfit
in high-level context information, and ignore the atomic action details in the
video. Second, the coarse action classes often lead to the ambiguous
annotations of temporal boundaries, which are inappropriate for temporal action
localization. To tackle these problems, we develop a novel large-scale and
fine-grained video dataset, coined as FineAction, for temporal action
localization. In total, FineAction contains 103K temporal instances of 106
action categories, annotated in 17K untrimmed videos. Compared to the existing
TAL datasets, our FineAction takes distinct characteristics of fine action
classes with rich diversity, dense annotations of multiple instances, and
co-occurring actions of different classes, which introduces new opportunities
and challenges for temporal action localization. To benchmark FineAction, we
systematically investigate the performance of several popular temporal
localization methods on it, and deeply analyze the influence of fine-grained
instances in temporal action localization. As a minor contribution, we present
a simple baseline approach for handling the fine-grained action detection,
which achieves an mAP of 13.17% on our FineAction. We believe that FineAction
can advance research of temporal action localization and beyond.
