  Face recognition service has been used in many fields and brings much
convenience to people. However, once the user's facial data is transmitted to a
service provider, the user will lose control of his/her private data. In recent
years, there exist various security and privacy issues due to the leakage of
facial data. Although many privacy-preserving methods have been proposed, they
usually fail when they are not accessible to adversaries' strategies or
auxiliary data. Hence, in this paper, by fully considering two cases of
uploading facial images and facial features, which are very typical in face
recognition service systems, we proposed a data privacy minimization
transformation (PMT) method. This method can process the original facial data
based on the shallow model of authorized services to obtain the obfuscated
data. The obfuscated data can not only maintain satisfactory performance on
authorized models and restrict the performance on other unauthorized models but
also prevent original privacy data from leaking by AI methods and human visual
theft. Additionally, since a service provider may execute preprocessing
operations on the received data, we also propose an enhanced perturbation
method to improve the robustness of PMT. Besides, to authorize one facial image
to multiple service models simultaneously, a multiple restriction mechanism is
proposed to improve the scalability of PMT. Finally, we conduct extensive
experiments and evaluate the effectiveness of the proposed PMT in defending
against face reconstruction, data abuse, and face attribute estimation attacks.
These experimental results demonstrate that PMT performs well in preventing
facial data abuse and privacy leakage while maintaining face recognition
accuracy.
