  Drones, or general UAVs, equipped with a single camera have been widely
deployed to a broad range of applications, such as aerial photography, fast
goods delivery and most importantly, surveillance. Despite the great progress
achieved in computer vision algorithms, these algorithms are not usually
optimized for dealing with images or video sequences acquired by drones, due to
various challenges such as occlusion, fast camera motion and pose variation. In
this paper, a drone-based multi-object tracking and 3D localization scheme is
proposed based on the deep learning based object detection. We first combine a
multi-object tracking method called TrackletNet Tracker (TNT) which utilizes
temporal and appearance information to track detected objects located on the
ground for UAV applications. Then, we are also able to localize the tracked
ground objects based on the group plane estimated from the Multi-View Stereo
technique. The system deployed on the drone can not only detect and track the
objects in a scene, but can also localize their 3D coordinates in meters with
respect to the drone camera. The experiments have proved our tracker can
reliably handle most of the detected objects captured by drones and achieve
favorable 3D localization performance when compared with the state-of-the-art
methods.
