  The neural network (NN) becomes one of the most heated type of models in
various signal processing applications. However, NNs are extremely vulnerable
to adversarial examples (AEs). To defend AEs, adversarial training (AT) is
believed to be the most effective method while due to the intensive
computation, AT is limited to be applied in most applications. In this paper,
to resolve the problem, we design a generic and efficient AT improvement
scheme, namely case-aware adversarial training (CAT). Specifically, the
intuition stems from the fact that a very limited part of informative samples
can contribute to most of model performance. Alternatively, if only the most
informative AEs are used in AT, we can lower the computation complexity of AT
significantly as maintaining the defense effect. To achieve this, CAT achieves
two breakthroughs. First, a method to estimate the information degree of
adversarial examples is proposed for AE filtering. Second, to further enrich
the information that the NN can obtain from AEs, CAT involves a weight
estimation and class-level balancing based sampling strategy to increase the
diversity of AT at each iteration. Extensive experiments show that CAT is
faster than vanilla AT by up to 3x while achieving competitive defense effect.
