  Multi-task missions for unmanned aerial vehicles (UAVs) involving inspection
and landing tasks are challenging for novice pilots due to the difficulties
associated with depth perception and the control interface. We propose a shared
autonomy system, alongside supplementary information displays, to assist pilots
to successfully complete multi-task missions without any pilot training. Our
approach comprises of three modules: (1) a perception module that encodes
visual information onto a latent representation, (2) a policy module that
augments pilot's actions, and (3) an information augmentation module that
provides additional information to the pilot. The policy module is trained in
simulation with simulated users and transferred to the real world without
modification in a user study (n=29), alongside supplementary information
schemes including learnt red/green light feedback cues and an augmented reality
display. The pilot's intent is unknown to the policy module and is inferred
from the pilot's input and UAV's states. The assistant increased task success
rate for the landing and inspection tasks from [16.67% & 54.29%] respectively
to [95.59% & 96.22%]. With the assistant, inexperienced pilots achieved similar
performance to experienced pilots. Red/green light feedback cues reduced the
required time by 19.53% and trajectory length by 17.86% for the inspection
task, where participants rated it as their preferred condition due to the
intuitive interface and providing reassurance. This work demonstrates that
simple user models can train shared autonomy systems in simulation, and
transfer to physical tasks to estimate user intent and provide effective
assistance and information to the pilot.
