  Class-conditional generative models hold promise to overcome the shortcomings
of their discriminative counterparts. They are a natural choice to solve
discriminative tasks in a robust manner as they jointly optimize for predictive
performance and accurate modeling of the input distribution. In this work, we
investigate robust classification with likelihood-based generative models from
a theoretical and practical perspective to investigate if they can deliver on
their promises. Our analysis focuses on a spectrum of robustness properties:
(1) Detection of worst-case outliers in the form of adversarial examples; (2)
Detection of average-case outliers in the form of ambiguous inputs and (3)
Detection of incorrectly labeled in-distribution inputs. Our theoretical result
reveals that it is impossible to guarantee detectability of
adversarially-perturbed inputs even for near-optimal generative classifiers.
Experimentally, we find that while we are able to train robust models for
MNIST, robustness completely breaks down on CIFAR10. We relate this failure to
various undesirable model properties that can be traced to the maximum
likelihood training objective. Despite being a common choice in the literature,
our results indicate that likelihood-based conditional generative models may
are surprisingly ineffective for robust classification.
