  Object detection and object tracking are usually treated as two separate
processes. Significant progress has been made for object detection in 2D images
using deep learning networks. The usual tracking-by-detection pipeline for
object tracking requires that the object is successfully detected in the first
frame and all subsequent frames, and tracking is done by associating detection
results. Performing object detection and object tracking through a single
network remains a challenging open question. We propose a novel network
structure named trackNet that can directly detect a 3D tube enclosing a moving
object in a video segment by extending the faster R-CNN framework. A Tube
Proposal Network (TPN) inside the trackNet is proposed to predict the
objectness of each candidate tube and location parameters specifying the
bounding tube. The proposed framework is applicable for detecting and tracking
any object and in this paper, we focus on its application for traffic video
analysis. The proposed model is trained and tested on UA-DETRAC, a large
traffic video dataset available for multi-vehicle detection and tracking, and
obtained very promising results.
