  Technological advancements have normalized the usage of unmanned aerial
vehicles (UAVs) in every sector, spanning from military to commercial but they
also pose serious security concerns due to their enhanced functionalities and
easy access to private and highly secured areas. Several instances related to
UAVs have raised security concerns, leading to UAV detection research studies.
Visual techniques are widely adopted for UAV detection, but they perform poorly
at night, in complex backgrounds, and in adverse weather conditions. Therefore,
a robust night vision-based drone detection system is required to that could
efficiently tackle this problem. Infrared cameras are increasingly used for
nighttime surveillance due to their wide applications in night vision
equipment. This paper uses a deep learning-based TinyFeatureNet (TF-Net), which
is an improved version of YOLOv5s, to accurately detect UAVs during the night
using infrared (IR) images. In the proposed TF-Net, we introduce architectural
changes in the neck and backbone of the YOLOv5s. We also simulated four
different YOLOv5 models (s,m,n,l) and proposed TF-Net for a fair comparison.
The results showed better performance for the proposed TF-Net in terms of
precision, IoU, GFLOPS, model size, and FPS compared to the YOLOv5s. TF-Net
yielded the best results with 95.7\% precision, 84\% mAp, and 44.8\% $IoU$.
