  Deep neural networks have achieved substantial achievements in several
computer vision areas, but have vulnerabilities that are often fooled by
adversarial examples that are not recognized by humans. This is an important
issue for security or medical applications. In this paper, we propose an
ensemble model training framework with random layer sampling to improve the
robustness of deep neural networks. In the proposed training framework, we
generate various sampled model through the random layer sampling and update the
weight of the sampled model. After the ensemble models are trained, it can hide
the gradient efficiently and avoid the gradient-based attack by the random
layer sampling method. To evaluate our proposed method, comprehensive and
comparative experiments have been conducted on three datasets. Experimental
results show that the proposed method improves the adversarial robustness.
