  The separation between training and deployment of machine learning models
implies that not all scenarios encountered in deployment can be anticipated
during training, and therefore relying solely on advancements in training has
its limits. Out-of-distribution (OOD) detection is an important area that
stress-tests a model's ability to handle unseen situations: Do models know when
they don't know? Existing OOD detection methods either incur extra training
steps, additional data or make nontrivial modifications to the trained network.
In contrast, in this work, we propose an extremely simple, post-hoc, on-the-fly
activation shaping method, ASH, where a large portion (e.g. 90%) of a sample's
activation at a late layer is removed, and the rest (e.g. 10%) simplified or
lightly adjusted. The shaping is applied at inference time, and does not
require any statistics calculated from training data. Experiments show that
such a simple treatment enhances in-distribution and out-of-distribution
distinction so as to allow state-of-the-art OOD detection on ImageNet, and does
not noticeably deteriorate the in-distribution accuracy. Video, animation and
code can be found at: https://andrijazz.github.io/ash
