  Out-of-distribution (OOD) detection plays a crucial role in ensuring the
safety and reliability of deep neural networks in various applications. While
there has been a growing focus on OOD detection in visual data, the field of
textual OOD detection has received less attention. Only a few attempts have
been made to directly apply general OOD detection methods to natural language
processing (NLP) tasks, without adequately considering the characteristics of
textual data. In this paper, we delve into textual OOD detection with
Transformers. We first identify a key problem prevalent in existing OOD
detection methods: the biased representation learned through the maximization
of the conditional likelihood $p(y\mid x)$ can potentially result in subpar
performance. We then propose a novel variational inference framework for OOD
detection (VI-OOD), which maximizes the likelihood of the joint distribution
$p(x, y)$ instead of $p(y\mid x)$. VI-OOD is tailored for textual OOD detection
by efficiently exploiting the representations of pre-trained Transformers.
Through comprehensive experiments on various text classification tasks, VI-OOD
demonstrates its effectiveness and wide applicability. Our code has been
released at \url{https://github.com/liam0949/LLM-OOD}.
