  Pretrained Transformers achieve remarkable performance when training and test
data are from the same distribution. However, in real-world scenarios, the
model often faces out-of-distribution (OOD) instances that can cause severe
semantic shift problems at inference time. Therefore, in practice, a reliable
model should identify such instances, and then either reject them during
inference or pass them over to models that handle another distribution. In this
paper, we develop an unsupervised OOD detection method, in which only the
in-distribution (ID) data are used in training. We propose to fine-tune the
Transformers with a contrastive loss, which improves the compactness of
representations, such that OOD instances can be better differentiated from ID
ones. These OOD instances can then be accurately detected using the Mahalanobis
distance in the model's penultimate layer. We experiment with comprehensive
settings and achieve near-perfect OOD detection performance, outperforming
baselines drastically. We further investigate the rationales behind the
improvement, finding that more compact representations through margin-based
contrastive learning bring the improvement. We release our code to the
community for future research.
