  Modern machine learning models deployed in the wild can encounter both
covariate and semantic shifts, giving rise to the problems of
out-of-distribution (OOD) generalization and OOD detection respectively. While
both problems have received significant research attention lately, they have
been pursued independently. This may not be surprising, since the two tasks
have seemingly conflicting goals. This paper provides a new unified approach
that is capable of simultaneously generalizing to covariate shifts while
robustly detecting semantic shifts. We propose a margin-based learning
framework that exploits freely available unlabeled data in the wild that
captures the environmental test-time OOD distributions under both covariate and
semantic shifts. We show both empirically and theoretically that the proposed
margin constraint is the key to achieving both OOD generalization and
detection. Extensive experiments show the superiority of our framework,
outperforming competitive baselines that specialize in either OOD
generalization or OOD detection. Code is publicly available at
https://github.com/deeplearning-wisc/scone.
