  Applying encryption technology to image retrieval can ensure the security and
privacy of personal images. The related researches in this field have focused
on the organic combination of encryption algorithm and artificial feature
extraction. Many existing encrypted image retrieval schemes cannot prevent
feature leakage and file size increase or cannot achieve satisfied retrieval
performance. In this paper, A new end-to-end encrypted image retrieval scheme
is presented. First, images are encrypted by using block rotation, new
orthogonal transforms and block permutation during the JPEG compression
process. Second, we combine the triplet loss and the cross entropy loss to
train a network model, which contains gMLP modules, by end-to-end learning for
extracting cipher-images' features. Compared with manual features extraction
such as extracting color histogram, the end-to-end mechanism can economize on
manpower. Experimental results show that our scheme has good retrieval
performance, while can ensure compression friendly and no feature leakage.
