  Object detection is increasingly used onboard Unmanned Aerial Vehicles (UAV)
for various applications; however, the machine learning (ML) models for
UAV-based detection are often validated using data curated for tasks unrelated
to the UAV application. This is a concern because training neural networks on
large-scale benchmarks have shown excellent capability in generic object
detection tasks, yet conventional training approaches can lead to large
inference errors for UAV-based images. Such errors arise due to differences in
imaging conditions between images from UAVs and images in training. To overcome
this problem, we characterize boundary conditions of ML models, beyond which
the models exhibit rapid degradation in detection accuracy. Our work is focused
on understanding the impact of different UAV-based imaging conditions on
detection performance by using synthetic data generated using a game engine.
Properties of the game engine are exploited to populate the synthetic datasets
with realistic and annotated images. Specifically, it enables the fine control
of various parameters, such as camera position, view angle, illumination
conditions, and object pose. Using the synthetic datasets, we analyze detection
accuracy in different imaging conditions as a function of the above parameters.
We use three well-known neural network models with different model complexity
in our work. In our experiment, we observe and quantify the following: 1) how
detection accuracy drops as the camera moves toward the nadir-view region; 2)
how detection accuracy varies depending on different object poses, and 3) the
degree to which the robustness of the models changes as illumination conditions
vary.
