  In this paper we consider the task of recognizing human actions in realistic
video where human actions are dominated by irrelevant factors. We first study
the benefits of removing non-action video segments, which are the ones that do
not portray any human action. We then learn a non-action classifier and use it
to down-weight irrelevant video segments. The non-action classifier is trained
using ActionThread, a dataset with shot-level annotation for the occurrence or
absence of a human action. The non-action classifier can be used to identify
non-action shots with high precision and subsequently used to improve the
performance of action recognition systems.
