  Autonomous Unmanned Aerial Vehicles (UAVs) have become essential tools in
defense, law enforcement, disaster response, and product delivery. These
autonomous navigation systems require a wireless communication network, and of
late are deep learning based. In critical scenarios such as border protection
or disaster response, ensuring the secure navigation of autonomous UAVs is
paramount. But, these autonomous UAVs are susceptible to adversarial attacks
through the communication network or the deep learning models - eavesdropping /
man-in-the-middle / membership inference / reconstruction. To address this
susceptibility, we propose an innovative approach that combines Reinforcement
Learning (RL) and Fully Homomorphic Encryption (FHE) for secure autonomous UAV
navigation. This end-to-end secure framework is designed for real-time video
feeds captured by UAV cameras and utilizes FHE to perform inference on
encrypted input images. While FHE allows computations on encrypted data,
certain computational operators are yet to be implemented. Convolutional neural
networks, fully connected neural networks, activation functions and OpenAI Gym
Library are meticulously adapted to the FHE domain to enable encrypted data
processing. We demonstrate the efficacy of our proposed approach through
extensive experimentation. Our proposed approach ensures security and privacy
in autonomous UAV navigation with negligible loss in performance.
