  Predicting the outcomes of integrating Unmanned Aerial Systems (UAS) into the
National Aerospace (NAS) is a complex problem which is required to be addressed
by simulation studies before allowing the routine access of UAS into the NAS.
This thesis focuses on providing 2D and 3D simulation frameworks using a game
theoretical methodology to evaluate integration concepts in scenarios where
manned and unmanned air vehicles co-exist. The fundamental gap in the
literature is that the models of interaction between manned and unmanned
vehicles are insufficient: a) they assume that pilot behavior is known a priori
and b) they disregard decision making processes. The contribution of this work
is to propose a modeling framework, in which, human pilot reactions are modeled
using reinforcement learning and a game theoretical concept called level-k
reasoning to fill this gap. The level-k reasoning concept is based on the
assumption that humans have various levels of decision making. Reinforcement
learning is a mathematical learning method that is rooted in human learning. In
this work, a classical and an approximate reinforcement learning (Neural Fitted
Q Iteration) methods are used to model time-extended decisions of pilots with
2D and 3D maneuvers. An analysis of UAS integration is conducted using example
scenarios in the presence of manned aircraft and fully autonomous UAS equipped
with sense and avoid algorithms.
