  The cooperation of a human pilot with an autonomous agent during flight
control realizes parallel autonomy. We propose an air-guardian system that
facilitates cooperation between a pilot with eye tracking and a parallel
end-to-end neural control system. Our vision-based air-guardian system combines
a causal continuous-depth neural network model with a cooperation layer to
enable parallel autonomy between a pilot and a control system based on
perceived differences in their attention profiles. The attention profiles for
neural networks are obtained by computing the networks' saliency maps (feature
importance) through the VisualBackProp algorithm, while the attention profiles
for humans are either obtained by eye tracking of human pilots or saliency maps
of networks trained to imitate human pilots. When the attention profile of the
pilot and guardian agents align, the pilot makes control decisions. Otherwise,
the air-guardian makes interventions and takes over the control of the
aircraft. We show that our attention-based air-guardian system can balance the
trade-off between its level of involvement in the flight and the pilot's
expertise and attention. The guardian system is particularly effective in
situations where the pilot was distracted due to information overload. We
demonstrate the effectiveness of our method for navigating flight scenarios in
simulation with a fixed-wing aircraft and on hardware with a quadrotor
platform.
