  We present Vax-a-Net; a technique for immunizing convolutional neural
networks (CNNs) against adversarial patch attacks (APAs). APAs insert visually
overt, local regions (patches) into an image to induce misclassification. We
introduce a conditional Generative Adversarial Network (GAN) architecture that
simultaneously learns to synthesise patches for use in APAs, whilst exploiting
those attacks to adapt a pre-trained target CNN to reduce its susceptibility to
them. This approach enables resilience against APAs to be conferred to
pre-trained models, which would be impractical with conventional adversarial
training due to the slow convergence of APA methods. We demonstrate
transferability of this protection to defend against existing APAs, and show
its efficacy across several contemporary CNN architectures.
