  Out-of-Distribution (OoD) detection is important for building safe artificial
intelligence systems. However, current OoD detection methods still cannot meet
the performance requirements for practical deployment. In this paper, we
propose a simple yet effective algorithm based on a novel observation: in a
trained neural network, OoD samples with bounded norms well concentrate in the
feature space. We call the center of OoD features the Feature Space Singularity
(FSS), and denote the distance of a sample feature to FSS as FSSD. Then, OoD
samples can be identified by taking a threshold on the FSSD. Our analysis of
the phenomenon reveals why our algorithm works. We demonstrate that our
algorithm achieves state-of-the-art performance on various OoD detection
benchmarks. Besides, FSSD also enjoys robustness to slight corruption in test
data and can be further enhanced by ensembling. These make FSSD a promising
algorithm to be employed in real world. We release our code at
\url{https://github.com/megvii-research/FSSD_OoD_Detection}.
