Learning an Urban Air Mobility Encounter Model from Expert Preferences
  Airspace models have played an important role in the development and
evaluation of aircraft collision avoidance systems for both manned and unmanned
aircraft. As Urban Air Mobility (UAM) systems are being developed, we need new
encounter models that are representative of their operational environment.
Developing such models is challenging due to the lack of data on UAM behavior
in the airspace. While previous encounter models for other aircraft types rely
on large datasets to produce realistic trajectories, this paper presents an
approach to encounter modeling that instead relies on expert knowledge. In
particular, recent advances in preference-based learning are extended to tune
an encounter model from expert preferences. The model takes the form of a
stochastic policy for a Markov decision process (MDP) in which the reward
function is learned from pairwise queries of a domain expert. We evaluate the
performance of two querying methods that seek to maximize the information
obtained from each query. Ultimately, we demonstrate a method for generating
realistic encounter trajectories with only a few minutes of an expert's time.
