  The recent development of commodity 360$^{\circ}$ cameras have enabled a
single video to capture an entire scene, which endows promising potentials in
surveillance scenarios. However, research in omnidirectional video analysis has
lagged behind the hardware advances. In this work, we address the important
problem of action recognition in top-view 360$^{\circ}$ videos. Due to the wide
filed-of-view, 360$^{\circ}$ videos usually capture multiple people performing
actions at the same time. Furthermore, the appearance of people are deformed.
The proposed framework first transforms omnidirectional videos into panoramic
videos, then it extracts spatial-temporal features using region-based 3D CNNs
for action recognition. We propose a weakly-supervised method based on
multi-instance multi-label learning, which trains the model to recognize and
localize multiple actions in a video using only video-level action labels as
supervision. We perform experiments to quantitatively validate the efficacy of
the proposed method and qualitatively demonstrate action localization results.
To enable research in this direction, we introduce 360Action, the first
omnidirectional video dataset for multi-person action recognition.
