  As vision-language models like CLIP are widely applied to zero-shot tasks and
gain remarkable performance on in-distribution (ID) data, detecting and
rejecting out-of-distribution (OOD) inputs in the zero-shot setting have become
crucial for ensuring the safety of using such models on the fly. Most existing
zero-shot OOD detectors rely on ID class label-based prompts to guide CLIP in
classifying ID images and rejecting OOD images. In this work we instead propose
to leverage a large set of diverse auxiliary outlier class labels as pseudo OOD
class text prompts to CLIP for enhancing zero-shot OOD detection, an approach
we called Outlier Label Exposure (OLE). The key intuition is that ID images are
expected to have lower similarity to these outlier class prompts than OOD
images. One issue is that raw class labels often include noise labels, e.g.,
synonyms of ID labels, rendering raw OLE-based detection ineffective. To
address this issue, we introduce an outlier prototype learning module that
utilizes the prompt embeddings of the outlier labels to learn a small set of
pivotal outlier prototypes for an embedding similarity-based OOD scoring.
Additionally, the outlier classes and their prototypes can be loosely coupled
with the ID classes, leading to an inseparable decision region between them.
Thus, we also introduce an outlier label generation module that synthesizes our
outlier prototypes and ID class embeddings to generate in-between outlier
prototypes to further calibrate the detection in OLE. Despite its simplicity,
extensive experiments show that OLE substantially improves detection
performance and achieves new state-of-the-art performance in large-scale OOD
and hard OOD detection benchmarks.
