  We present a novel feature selection technique, Sparse Linear
Centroid-Encoder (SLCE). The algorithm uses a linear transformation to
reconstruct a point as its class centroid and, at the same time, uses the
$\ell_1$-norm penalty to filter out unnecessary features from the input data.
The original formulation of the optimization problem is nonconvex, but we
propose a two-step approach, where each step is convex. In the first step, we
solve the linear Centroid-Encoder, a convex optimization problem over a matrix
$A$. In the second step, we only search for a sparse solution over a diagonal
matrix $B$ while keeping $A$ fixed. Unlike other linear methods, e.g., Sparse
Support Vector Machines and Lasso, Sparse Linear Centroid-Encoder uses a single
model for multi-class data. We present an in-depth empirical analysis of the
proposed model and show that it promotes sparsity on various data sets,
including high-dimensional biological data. Our experimental results show that
SLCE has a performance advantage over some state-of-the-art neural
network-based feature selection techniques.
