  For real-world language applications, detecting an out-of-distribution (OOD)
sample is helpful to alert users or reject such unreliable samples. However,
modern over-parameterized language models often produce overconfident
predictions for both in-distribution (ID) and OOD samples. In particular,
language models suffer from OOD samples with a similar semantic representation
to ID samples since these OOD samples lie near the ID manifold. A rejection
network can be trained with ID and diverse outlier samples to detect test OOD
samples, but explicitly collecting auxiliary OOD datasets brings an additional
burden for data collection. In this paper, we propose a simple but effective
method called Pseudo Outlier Exposure (POE) that constructs a surrogate OOD
dataset by sequentially masking tokens related to ID classes. The surrogate OOD
sample introduced by POE shows a similar representation to ID data, which is
most effective in training a rejection network. Our method does not require any
external OOD data and can be easily implemented within off-the-shelf
Transformers. A comprehensive comparison with state-of-the-art algorithms
demonstrates POE's competitiveness on several text classification benchmarks.
