  Human behavior understanding with unmanned aerial vehicles (UAVs) is of great
significance for a wide range of applications, which simultaneously brings an
urgent demand of large, challenging, and comprehensive benchmarks for the
development and evaluation of UAV-based models. However, existing benchmarks
have limitations in terms of the amount of captured data, types of data
modalities, categories of provided tasks, and diversities of subjects and
environments. Here we propose a new benchmark - UAVHuman - for human behavior
understanding with UAVs, which contains 67,428 multi-modal video sequences and
119 subjects for action recognition, 22,476 frames for pose estimation, 41,290
frames and 1,144 identities for person re-identification, and 22,263 frames for
attribute recognition. Our dataset was collected by a flying UAV in multiple
urban and rural districts in both daytime and nighttime over three months,
hence covering extensive diversities w.r.t subjects, backgrounds,
illuminations, weathers, occlusions, camera motions, and UAV flying attitudes.
Such a comprehensive and challenging benchmark shall be able to promote the
research of UAV-based human behavior understanding, including action
recognition, pose estimation, re-identification, and attribute recognition.
Furthermore, we propose a fisheye-based action recognition method that
mitigates the distortions in fisheye videos via learning unbounded
transformations guided by flat RGB videos. Experiments show the efficacy of our
method on the UAV-Human dataset. The project page:
https://github.com/SUTDCV/UAV-Human
