  Deep learning methods have boosted the adoption of NLP systems in real-life
applications. However, they turn out to be vulnerable to distribution shifts
over time which may cause severe dysfunctions in production systems, urging
practitioners to develop tools to detect out-of-distribution (OOD) samples
through the lens of the neural network. In this paper, we introduce TRUSTED, a
new OOD detector for classifiers based on Transformer architectures that meets
operational requirements: it is unsupervised and fast to compute. The
efficiency of TRUSTED relies on the fruitful idea that all hidden layers carry
relevant information to detect OOD examples. Based on this, for a given input,
TRUSTED consists in (i) aggregating this information and (ii) computing a
similarity score by exploiting the training distribution, leveraging the
powerful concept of data depth. Our extensive numerical experiments involve 51k
model configurations, including various checkpoints, seeds, and datasets, and
demonstrate that TRUSTED achieves state-of-the-art performances. In particular,
it improves previous AUROC over 3 points.
