  Recent studies have shown that state-of-the-art deep learning models are
vulnerable to the inputs with small perturbations (adversarial examples). We
observe two critical obstacles in adversarial examples: (i) Strong adversarial
attacks (e.g., C&W attack) require manually tuning hyper-parameters and take a
long time to construct an adversarial example, making it impractical to attack
real-time systems; (ii) Most of the studies focus on non-sequential tasks, such
as image classification, yet only a few consider sequential tasks. In this
work, we speed up adversarial attacks, especially on sequential learning tasks.
By leveraging the uncertainty of each task, we directly learn the adaptive
multi-task weightings, without manually searching hyper-parameters. A unified
architecture is developed and evaluated for both non-sequential tasks and
sequential ones. To validate the effectiveness, we take the scene text
recognition task as a case study. To our best knowledge, our proposed method is
the first attempt to adversarial attack for scene text recognition. Adaptive
Attack achieves over 99.9\% success rate with 3-6X speedup compared to
state-of-the-art adversarial attacks.
