  The properties of individual neurons are often analyzed in order to
understand the biological and artificial neural networks in which they're
embedded. Class selectivity-typically defined as how different a neuron's
responses are across different classes of stimuli or data samples-is commonly
used for this purpose. However, it remains an open question whether it is
necessary and/or sufficient for deep neural networks (DNNs) to learn class
selectivity in individual units. We investigated the causal impact of class
selectivity on network function by directly regularizing for or against class
selectivity. Using this regularizer to reduce class selectivity across units in
convolutional neural networks increased test accuracy by over 2% for ResNet18
trained on Tiny ImageNet. For ResNet20 trained on CIFAR10 we could reduce class
selectivity by a factor of 2.5 with no impact on test accuracy, and reduce it
nearly to zero with only a small ($\sim$2%) drop in test accuracy. In contrast,
regularizing to increase class selectivity significantly decreased test
accuracy across all models and datasets. These results indicate that class
selectivity in individual units is neither sufficient nor strictly necessary,
and can even impair DNN performance. They also encourage caution when focusing
on the properties of single units as representative of the mechanisms by which
DNNs function.
