  Adversarial examples have posed a severe threat to deep neural networks due
to their transferable nature. Currently, various works have paid great efforts
to enhance the cross-model transferability, which mostly assume the substitute
model is trained in the same domain as the target model. However, in reality,
the relevant information of the deployed model is unlikely to leak. Hence, it
is vital to build a more practical black-box threat model to overcome this
limitation and evaluate the vulnerability of deployed models. In this paper,
with only the knowledge of the ImageNet domain, we propose a Beyond ImageNet
Attack (BIA) to investigate the transferability towards black-box domains
(unknown classification tasks). Specifically, we leverage a generative model to
learn the adversarial function for disrupting low-level features of input
images. Based on this framework, we further propose two variants to narrow the
gap between the source and target domains from the data and model perspectives,
respectively. Extensive experiments on coarse-grained and fine-grained domains
demonstrate the effectiveness of our proposed methods. Notably, our methods
outperform state-of-the-art approaches by up to 7.71\% (towards coarse-grained
domains) and 25.91\% (towards fine-grained domains) on average. Our code is
available at \url{https://github.com/qilong-zhang/Beyond-ImageNet-Attack}.
