  Private inference (PI) has emerged as a promising solution to execute
computations on encrypted data, safeguarding user privacy and model parameters
in edge computing. However, existing PI methods are predominantly developed
considering constant resource constraints, overlooking the varied and dynamic
resource constraints in diverse edge devices, like energy budgets.
Consequently, model providers have to design specialized models for different
devices, where all of them have to be stored on the edge server, resulting in
inefficient deployment. To fill this gap, this work presents AdaPI, a novel
approach that achieves adaptive PI by allowing a model to perform well across
edge devices with diverse energy budgets. AdaPI employs a PI-aware training
strategy that optimizes the model weights alongside weight-level and
feature-level soft masks. These soft masks are subsequently transformed into
multiple binary masks to enable adjustments in communication and computation
workloads. Through sequentially training the model with increasingly dense
binary masks, AdaPI attains optimal accuracy for each energy budget, which
outperforms the state-of-the-art PI methods by 7.3\% in terms of test accuracy
on CIFAR-100. The code of AdaPI can be accessed via
https://github.com/jiahuiiiiii/AdaPI.
