  Autoencoders have been widely used as a nonlinear tool for data
dimensionality reduction. While autoencoders don't utilize the label
information, Centroid-Encoders (CE)\cite{ghosh2022supervised} use the class
label in their learning process. In this study, we propose a sparse
optimization using the Centroid-Encoder architecture to determine a minimal set
of features that discriminate between two or more classes. The resulting
algorithm, Sparse Centroid-Encoder (SCE), extracts discriminatory features in
groups using a sparsity inducing $\ell_1$-norm while mapping a point to its
class centroid. One key attribute of SCE is that it can extract informative
features from a multi-modal data set, i.e., data sets whose classes appear to
have multiple clusters. The algorithm is applied to a wide variety of real
world data sets, including single-cell data, high dimensional biological data,
image data, speech data, and accelerometer sensor data. We compared our method
to various state-of-the-art feature selection techniques, including supervised
Concrete Autoencoders (SCAE), Feature Selection Network (FsNet), deep feature
selection (DFS), Stochastic Gate (STG), and LassoNet. We empirically showed
that SCE features often produced better classification accuracy than other
methods on sequester test set.
