  Protecting privacy during inference with deep neural networks is possible by
adding noise to the activations in the last layers prior to the final
classifiers or other task-specific layers. The activations in such layers are
known as "features" (or, less commonly, as "embeddings" or "feature
embeddings"). The added noise helps prevent reconstruction of the inputs from
the noisy features. Lower bounding the variance of every possible unbiased
estimator of the inputs quantifies the confidentiality arising from such added
noise. Convenient, computationally tractable bounds are available from classic
inequalities of Hammersley and of Chapman and Robbins -- the HCR bounds.
Numerical experiments indicate that the HCR bounds are on the precipice of
being effectual for small neural nets with the data sets, "MNIST" and
"CIFAR-10," which contain 10 classes each for image classification. The HCR
bounds appear to be insufficient on their own to guarantee confidentiality of
the inputs to inference with standard deep neural nets, "ResNet-18" and
"Swin-T," pre-trained on the data set, "ImageNet-1000," which contains 1000
classes. Supplementing the addition of noise to features with other methods for
providing confidentiality may be warranted in the case of ImageNet. In all
cases, the results reported here limit consideration to amounts of added noise
that incur little degradation in the accuracy of classification from the noisy
features. Thus, the added noise enhances confidentiality without much reduction
in the accuracy on the task of image classification.
