  Out-of-distribution (OOD) detection methods often exploit auxiliary outliers
to train model identifying OOD samples, especially discovering challenging
outliers from auxiliary outliers dataset to improve OOD detection. However,
they may still face limitations in effectively distinguishing between the most
challenging OOD samples that are much like in-distribution (ID) data, i.e.,
\idlike samples. To this end, we propose a novel OOD detection framework that
discovers \idlike outliers using CLIP \cite{DBLP:conf/icml/RadfordKHRGASAM21}
from the vicinity space of the ID samples, thus helping to identify these most
challenging OOD samples. Then a prompt learning framework is proposed that
utilizes the identified \idlike outliers to further leverage the capabilities
of CLIP for OOD detection. Benefiting from the powerful CLIP, we only need a
small number of ID samples to learn the prompts of the model without exposing
other auxiliary outlier datasets. By focusing on the most challenging \idlike
OOD samples and elegantly exploiting the capabilities of CLIP, our method
achieves superior few-shot learning performance on various real-world image
datasets (e.g., in 4-shot OOD detection on the ImageNet-1k dataset, our method
reduces the average FPR95 by 12.16\% and improves the average AUROC by 2.76\%,
compared to state-of-the-art methods). Code is available at
https://github.com/ycfate/ID-like.
