  This paper adds to the fundamental body of work on benchmarking the
robustness of deep learning (DL) classifiers. We innovate a new benchmarking
methodology to evaluate robustness of DL classifiers. Also, we introduce a new
four-quadrant statistical visualization tool, including minimum accuracy,
maximum accuracy, mean accuracy, and coefficient of variation, for benchmarking
robustness of DL classifiers. To measure robust DL classifiers, we created a
comprehensive 69 benchmarking image set, including a clean set, sets with
single factor perturbations, and sets with two-factor perturbation conditions.
After collecting experimental results, we first report that using two-factor
perturbed images improves both robustness and accuracy of DL classifiers. The
two-factor perturbation includes (1) two digital perturbations (salt & pepper
noise and Gaussian noise) applied in both sequences, and (2) one digital
perturbation (salt & pepper noise) and a geometric perturbation (rotation)
applied in both sequences. All source codes, related image sets, and
preliminary data, figures are shared on a GitHub website to support future
academic research and industry projects. The web resources locate at
https://github.com/caperock/robustai
