  Adversarial transferability in black-box scenarios presents a unique
challenge: while attackers can employ surrogate models to craft adversarial
examples, they lack assurance on whether these examples will successfully
compromise the target model. Until now, the prevalent method to ascertain
success has been trial and error-testing crafted samples directly on the victim
model. This approach, however, risks detection with every attempt, forcing
attackers to either perfect their first try or face exposure. Our paper
introduces a ranking strategy that refines the transfer attack process,
enabling the attacker to estimate the likelihood of success without repeated
trials on the victim's system. By leveraging a set of diverse surrogate models,
our method can predict transferability of adversarial examples. This strategy
can be used to either select the best sample to use in an attack or the best
perturbation to apply to a specific sample. Using our strategy, we were able to
raise the transferability of adversarial examples from a mere 20% - akin to
random selection-up to near upper-bound levels, with some scenarios even
witnessing a 100% success rate. This substantial improvement not only sheds
light on the shared susceptibilities across diverse architectures but also
demonstrates that attackers can forego the detectable trial-and-error tactics
raising increasing the threat of surrogate-based attacks.
