  The majority of adversarial attack techniques perform well against deep face
recognition when the full knowledge of the system is revealed
(\emph{white-box}). However, such techniques act unsuccessfully in the gray-box
setting where the face templates are unknown to the attackers. In this work, we
propose a similarity-based gray-box adversarial attack (SGADV) technique with a
newly developed objective function. SGADV utilizes the dissimilarity score to
produce the optimized adversarial example, i.e., similarity-based adversarial
attack. This technique applies to both white-box and gray-box attacks against
authentication systems that determine genuine or imposter users using the
dissimilarity score. To validate the effectiveness of SGADV, we conduct
extensive experiments on face datasets of LFW, CelebA, and CelebA-HQ against
deep face recognition models of FaceNet and InsightFace in both white-box and
gray-box settings. The results suggest that the proposed method significantly
outperforms the existing adversarial attack techniques in the gray-box setting.
We hence summarize that the similarity-base approaches to develop the
adversarial example could satisfactorily cater to the gray-box attack scenarios
for de-authentication.
