  In this paper we consider high-dimensional multiclass classification by
sparse multinomial logistic regression. We propose first a feature selection
procedure based on penalized maximum likelihood with a complexity penalty on
the model size and derive the nonasymptotic bounds for misclassification excess
risk of the resulting classifier. We establish also their tightness by deriving
the corresponding minimax lower bounds. In particular, we show that there exist
two regimes corresponding to small and large number of classes. The bounds can
be reduced under the additional low noise condition. To find a penalized
maximum likelihood solution with a complexity penalty requires, however, a
combinatorial search over all possible models. To design a feature selection
procedure computationally feasible for high-dimensional data, we propose
multinomial logistic group Lasso and Slope classifiers and show that they also
achieve the minimax order.
