  In this work, we present a method to predict an entire `action tube' (a set
of temporally linked bounding boxes) in a trimmed video just by observing a
smaller subset of it. Predicting where an action is going to take place in the
near future is essential to many computer vision based applications such as
autonomous driving or surgical robotics. Importantly, it has to be done in
real-time and in an online fashion. We propose a Tube Prediction network
(TPnet) which jointly predicts the past, present and future bounding boxes
along with their action classification scores. At test time TPnet is used in a
(temporal) sliding window setting, and its predictions are put into a tube
estimation framework to construct/predict the video long action tubes not only
for the observed part of the video but also for the unobserved part.
Additionally, the proposed action tube predictor helps in completing action
tubes for unobserved segments of the video. We quantitatively demonstrate the
latter ability, and the fact that TPnet improves state-of-the-art detection
performance, on one of the standard action detection benchmarks - J-HMDB-21
dataset.
