  Wide Area Motion Imagery (WAMI) yields high-resolution images with a large
number of extremely small objects. Target objects have large spatial
displacements throughout consecutive frames. This nature of WAMI images makes
object tracking and detection challenging. In this paper, we present our deep
neural network-based combined object detection and tracking model, namely, Heat
Map Network (HM-Net). HM-Net is significantly faster than state-of-the-art
frame differencing and background subtraction-based methods, without
compromising detection and tracking performances. HM-Net follows the object
center-based joint detection and tracking paradigm. Simple heat map-based
predictions support an unlimited number of simultaneous detections. The
proposed method uses two consecutive frames and the object detection heat map
obtained from the previous frame as input, which helps HM-Net monitor
spatio-temporal changes between frames and keeps track of previously predicted
objects. Although reuse of prior object detection heat map acts as a vital
feedback-based memory element, it can lead to an unintended surge of
false-positive detections. To increase the robustness of the method against
false positives and to eliminate low confidence detections, HM-Net employs
novel feedback filters and advanced data augmentations. HM-Net outperforms
state-of-the-art WAMI moving object detection and tracking methods on the WPAFB
dataset with its 96.2% F1 and 94.4% mAP detection scores while achieving a
61.8% mAP tracking score on the same dataset. This performance corresponds to
an improvement of 2.1% for F1, 6.1% for mAP scores on detection, and 9.5% for
mAP score on tracking over the state-of-the-art.
