  New capabilities in wireless network security have been enabled by deep
learning, which leverages patterns in radio frequency (RF) data to identify and
authenticate devices. Open-set detection is an area of deep learning that
identifies samples captured from new devices during deployment that were not
part of the training set. Past work in open-set detection has mostly been
applied to independent and identically distributed data such as images. In
contrast, RF signal data present a unique set of challenges as the data forms a
time series with non-linear time dependencies among the samples. We introduce a
novel open-set detection approach based on the patterns of the hidden state
values within a Convolutional Neural Network (CNN) Long Short-Term Memory
(LSTM) model. Our approach greatly improves the Area Under the Precision-Recall
Curve on LoRa, Wireless-WiFi, and Wired-WiFi datasets, and hence, can be used
successfully to monitor and control unauthorized network access of wireless
devices.
