  We introduce a novel nonlinear model, Sparse Adaptive Bottleneck
Centroid-Encoder (SABCE), for determining the features that discriminate
between two or more classes. The algorithm aims to extract discriminatory
features in groups while reconstructing the class centroids in the ambient
space and simultaneously use additional penalty terms in the bottleneck layer
to decrease within-class scatter and increase the separation of different class
centroids. The model has a sparsity-promoting layer (SPL) with a one-to-one
connection to the input layer. Along with the primary objective, we minimize
the $l_{2,1}$-norm of the sparse layer, which filters out unnecessary features
from input data. During training, we update class centroids by taking the
Hadamard product of the centroids and weights of the sparse layer, thus
ignoring the irrelevant features from the target. Therefore the proposed method
learns to reconstruct the critical components of class centroids rather than
the whole centroids. The algorithm is applied to various real-world data sets,
including high-dimensional biological, image, speech, and accelerometer sensor
data. We compared our method to different state-of-the-art feature selection
techniques, including supervised Concrete Autoencoders (SCAE), Feature
Selection Networks (FsNet), Stochastic Gates (STG), and LassoNet. We
empirically showed that SABCE features often produced better classification
accuracy than other methods on the sequester test sets, setting new
state-of-the-art results.
