  In this paper, we focus on the potential use of unmanned aerial vehicles
(UAVs) for search and rescue (SAR) missions in GPS-denied indoor environments.
We consider the problem of navigating a UAV to a wireless signal source, e.g.,
a smartphone or watch owned by a victim. We assume that the source periodically
transmits RF signals to nearby wireless access points. Received signal strength
(RSS) at the UAV, which is a function of the UAV and source positions, is fed
to a Q-learning algorithm and the UAV is navigated to the vicinity of the
source. Unlike the traditional location-based Q-learning approach that uses the
GPS coordinates of the agent, our method uses the RSS to define the states and
rewards of the algorithm. It does not require any a priori information about
the environment. These, in turn, make it possible to use the UAVs in indoor SAR
operations. Two indoor scenarios with different dimensions are created using a
ray tracing software. Then, the corresponding heat maps that show the RSS at
each possible UAV location are extracted for more realistic analysis.
Performance of the RSS-based Q-learning algorithm is compared with the baseline
(location-based) Q-learning algorithm in terms of convergence speed, average
number of steps per episode, and the total length of the final trajectory. Our
results show that the RSS-based Q-learning provides competitive performance
with the location-based Q-learning.
