  Despite apparent human-level performances of deep neural networks (DNN), they
behave fundamentally differently from humans. They easily change predictions
when small corruptions such as blur and noise are applied on the input (lack of
robustness), and they often produce confident predictions on
out-of-distribution samples (improper uncertainty measure). While a number of
researches have aimed to address those issues, proposed solutions are typically
expensive and complicated (e.g. Bayesian inference and adversarial training).
Meanwhile, many simple and cheap regularization methods have been developed to
enhance the generalization of classifiers. Such regularization methods have
largely been overlooked as baselines for addressing the robustness and
uncertainty issues, as they are not specifically designed for that. In this
paper, we provide extensive empirical evaluations on the robustness and
uncertainty estimates of image classifiers (CIFAR-100 and ImageNet) trained
with state-of-the-art regularization methods. Furthermore, experimental results
show that certain regularization methods can serve as strong baseline methods
for robustness and uncertainty estimation of DNNs.
