  Massive human-related data is collected to train neural networks for computer
vision tasks. A major conflict is exposed relating to software engineers
between better developing AI systems and distancing from the sensitive training
data. To reconcile this conflict, this paper proposes an efficient
privacy-preserving learning paradigm, where images are first encrypted to
become ``human-imperceptible, machine-recognizable'' via one of the two
encryption strategies: (1) random shuffling to a set of equally-sized patches
and (2) mixing-up sub-patches of the images. Then, minimal adaptations are made
to vision transformer to enable it to learn on the encrypted images for vision
tasks, including image classification and object detection. Extensive
experiments on ImageNet and COCO show that the proposed paradigm achieves
comparable accuracy with the competitive methods. Decrypting the encrypted
images requires solving an NP-hard jigsaw puzzle or an ill-posed inverse
problem, which is empirically shown intractable to be recovered by various
attackers, including the powerful vision transformer-based attacker. We thus
show that the proposed paradigm can ensure the encrypted images have become
human-imperceptible while preserving machine-recognizable information. The code
is available at \url{https://github.com/FushengHao/PrivacyPreservingML.}
