  Deep neural network (DNN) models have become prevalent in edge devices for
real-time inference. However, they are vulnerable to model extraction attacks
and require protection. Existing defense approaches either fail to fully
safeguard model confidentiality or result in significant latency issues. To
overcome these challenges, this paper presents MirrorNet, which leverages
Trusted Execution Environment (TEE) to enable secure on-device DNN inference.
It generates a TEE-friendly implementation for any given DNN model to protect
the model confidentiality, while meeting the stringent computation and storage
constraints of TEE. The framework consists of two key components: the backbone
model (BackboneNet), which is stored in the normal world but achieves lower
inference accuracy, and the Companion Partial Monitor (CPM), a lightweight
mirrored branch stored in the secure world, preserving model confidentiality.
During inference, the CPM monitors the intermediate results from the
BackboneNet and rectifies the classification output to achieve higher accuracy.
To enhance flexibility, MirrorNet incorporates two modules: the CPM Strategy
Generator, which generates various protection strategies, and the Performance
Emulator, which estimates the performance of each strategy and selects the most
optimal one. Extensive experiments demonstrate the effectiveness of MirrorNet
in providing security guarantees while maintaining low computation latency,
making MirrorNet a practical and promising solution for secure on-device DNN
inference. For the evaluation, MirrorNet can achieve a 18.6% accuracy gap
between authenticated and illegal use, while only introducing 0.99% hardware
overhead.
