  Nowadays, deep learning methods, especially the convolutional neural networks
(CNNs), have shown impressive performance on extracting abstract and high-level
features from the hyperspectral image. However, general training process of
CNNs mainly considers the pixel-wise information or the samples' correlation to
formulate the penalization while ignores the statistical properties especially
the spectral variability of each class in the hyperspectral image. These
samples-based penalizations would lead to the uncertainty of the training
process due to the imbalanced and limited number of training samples. To
overcome this problem, this work characterizes each class from the
hyperspectral image as a statistical distribution and further develops a novel
statistical loss with the distributions, not directly with samples for deep
learning. Based on the Fisher discrimination criterion, the loss penalizes the
sample variance of each class distribution to decrease the intra-class variance
of the training samples. Moreover, an additional diversity-promoting condition
is added to enlarge the inter-class variance between different class
distributions and this could better discriminate samples from different classes
in hyperspectral image. Finally, the statistical estimation form of the
statistical loss is developed with the training samples through multi-variant
statistical analysis. Experiments over the real-world hyperspectral images show
the effectiveness of the developed statistical loss for deep learning.
