  This paper proposes a space-time multi-scale attention network (STANet) to
solve density map estimation, localization and tracking in dense crowds of
video clips captured by drones with arbitrary crowd density, perspective, and
flight altitude. Our STANet method aggregates multi-scale feature maps in
sequential frames to exploit the temporal coherency, and then predict the
density maps, localize the targets, and associate them in crowds
simultaneously. A coarse-to-fine process is designed to gradually apply the
attention module on the aggregated multi-scale feature maps to enforce the
network to exploit the discriminative space-time features for better
performance. The whole network is trained in an end-to-end manner with the
multi-task loss, formed by three terms, i.e., the density map loss,
localization loss and association loss. The non-maximal suppression followed by
the min-cost flow framework is used to generate the trajectories of targets' in
scenarios. Since existing crowd counting datasets merely focus on crowd
counting in static cameras rather than density map estimation, counting and
tracking in crowds on drones, we have collected a new large-scale drone-based
dataset, DroneCrowd, formed by 112 video clips with 33,600 high resolution
frames (i.e., 1920x1080) captured in 70 different scenarios. With intensive
amount of effort, our dataset provides 20,800 people trajectories with 4.8
million head annotations and several video-level attributes in sequences.
Extensive experiments are conducted on two challenging public datasets, i.e.,
Shanghaitech and UCF-QNRF, and our DroneCrowd, to demonstrate that STANet
achieves favorable performance against the state-of-the-arts. The datasets and
codes can be found at https://github.com/VisDrone.
