  Online Action Detection (OAD) in videos is proposed as a per-frame labeling
task to address the real-time prediction tasks that can only obtain the
previous and current video frames. This paper presents a novel
learning-with-privileged based framework for online action detection where the
future frames only observable at the training stages are considered as a form
of privileged information. Knowledge distillation is employed to transfer the
privileged information from the offline teacher to the online student. We note
that this setting is different from conventional KD because the difference
between the teacher and student models mostly lies in input data rather than
the network architecture. We propose Privileged Knowledge Distillation (PKD)
which (i) schedules a curriculum learning procedure and (ii) inserts auxiliary
nodes to the student model, both for shrinking the information gap and
improving learning performance. Compared to other OAD methods that explicitly
predict future frames, our approach avoids learning unpredictable unnecessary
yet inconsistent visual contents and achieves state-of-the-art accuracy on two
popular OAD benchmarks, TVSeries and THUMOS14.
