  Most visual retrieval applications store feature vectors for downstream
matching tasks. These vectors, from where user information can be spied out,
will cause privacy leakage if not carefully protected. To mitigate privacy
risks, current works primarily utilize non-invertible transformations or fully
cryptographic algorithms. However, transformation-based methods usually fail to
achieve satisfying matching performances while cryptosystems suffer from heavy
computational overheads. In addition, secure levels of current methods should
be improved to confront potential adversary attacks. To address these issues,
this paper proposes a plug-in module called SecureVector that protects features
by random permutations, 4L-DEC converting and existing homomorphic encryption
techniques. For the first time, SecureVector achieves real-time and lossless
feature matching among sanitized features, along with much higher security
levels than current state-of-the-arts. Extensive experiments on face
recognition, person re-identification, image retrieval, and privacy analyses
demonstrate the effectiveness of our method. Given limited public projects in
this field, codes of our method and implemented baselines are made open-source
in https://github.com/IrvingMeng/SecureVector.
