  In this thesis, we propose a light-weight sparsity-based algorithm, basic
thresholding classifier (BTC), for classification applications (such as face
identification, hyper-spectral image classification, etc.) which is capable of
identifying test samples extremely rapidly and performing high classification
accuracy. Originally BTC is a linear classifier which works based on the
assumption that the samples of the classes of a given dataset are linearly
separable. However, in practice those samples may not be linearly separable. In
this context, we also propose another algorithm namely kernel basic
thresholding classifier (KBTC) which is a non-linear kernel version of the BTC
algorithm. KBTC can achieve promising results especially when the given samples
are linearly non-separable. For both proposals, we introduce sufficient
identification conditions (SICs) under which BTC and KBTC can identify any test
sample in the range space of a given dictionary. By using SICs, we develop
parameter estimation procedures which do not require any cross validation. Both
BTC and KBTC algorithms provide efficient classifier fusion schemes in which
individual classifier outputs are combined to produce better classification
results. For instance, for the application of face identification, this is done
by combining the residuals having different random projectors. For spatial
applications such as hyper-spectral image classification, the fusion is carried
out by incorporating the spatial information, in which the output residual maps
are filtered using a smoothing filter. Numerical results on publicly available
face and hyper-spectral datasets show that our proposal outperforms well-known
support vector machines (SVM)-based techniques, multinomial logistic regression
(MLR)-based methods, and sparsity-based approaches like $l_1$-minimization and
simultaneous orthogonal matching pursuit (SOMP).
