  We consider high-dimensional multiclass classification by sparse multinomial
logistic regression. Unlike binary classification, in the multiclass setup one
can think about an entire spectrum of possible notions of sparsity associated
with different structural assumptions on the regression coefficients matrix. We
propose a computationally feasible feature selection procedure based on
penalized maximum likelihood with convex penalties capturing a specific type of
sparsity at hand. In particular, we consider global sparsity, double row-wise
sparsity, and low-rank sparsity, and show that with the properly chosen tuning
parameters the derived plug-in classifiers attain the minimax generalization
error bounds (in terms of misclassification excess risk) within the
corresponding classes of multiclass sparse linear classifiers. The developed
approach is general and can be adapted to other types of sparsity as well.
