  Medical images commonly exhibit multiple abnormalities. Predicting them
requires multi-class classifiers whose training and desired reliable
performance can be affected by a combination of factors, such as, dataset size,
data source, distribution, and the loss function used to train the deep neural
networks. Currently, the cross-entropy loss remains the de-facto loss function
for training deep learning classifiers. This loss function, however, asserts
equal learning from all classes, leading to a bias toward the majority class.
In this work, we benchmark various state-of-the-art loss functions that are
suitable for multi-class classification, critically analyze model performance,
and propose improved loss functions. We select a pediatric chest X-ray (CXR)
dataset that includes images with no abnormality (normal), and those exhibiting
manifestations consistent with bacterial and viral pneumonia. We construct
prediction-level and model-level ensembles, respectively, to improve
classification performance. Our results show that compared to the individual
models and the state-of-the-art literature, the weighted averaging of the
predictions for top-3 and top-5 model-level ensembles delivered significantly
superior classification performance (p < 0.05) in terms of MCC (0.9068, 95%
confidence interval (0.8839, 0.9297)) metric. Finally, we performed
localization studies to interpret model behaviors to visualize and confirm that
the individual models and ensembles learned meaningful features and highlighted
disease manifestations.
