  Out-of-distribution (OOD) detection is a rapidly growing field due to new
robustness and security requirements driven by an increased number of AI-based
systems. Existing OOD textual detectors often rely on an anomaly score (e.g.,
Mahalanobis distance) computed on the embedding output of the last layer of the
encoder. In this work, we observe that OOD detection performance varies greatly
depending on the task and layer output. More importantly, we show that the
usual choice (the last layer) is rarely the best one for OOD detection and that
far better results could be achieved if the best layer were picked. To leverage
this observation, we propose a data-driven, unsupervised method to combine
layer-wise anomaly scores. In addition, we extend classical textual OOD
benchmarks by including classification tasks with a greater number of classes
(up to 77), which reflects more realistic settings. On this augmented
benchmark, we show that the proposed post-aggregation methods achieve robust
and consistent results while removing manual feature selection altogether.
Their performance achieves near oracle's best layer performance.
