  Robustness of Deep Neural Networks (DNNs) is an important aspect to consider
for their clinical applications. This work examined robustness issue for a
DNN-based multi-class classification model via comprehensive experimental and
simulation studies. We constructed a DNN-based multi-class classification model
that classifies an axial CT image as one of the four body sites of brain & neck
(BN), chest (C), abdomen & pelvis (AP), and leg & foot (LF). The model was
trained with whole-body CT images of 37 patients each scanned once and 10 scans
of a whole-body phantom with different mAs levels to achieve a F1 score of
99.7% averaged over four classes on a testing dataset. We evaluated robustness
of the model against noise perturbations under different mAs levels using
simulated CT noisy images based on noise power spectrum (NPS) and experimental
CT images acquired in repeated scans of the orange-man phantom. To quantify
robustness of the trained model, we defined Successful Attack Rate (SAR) as the
ratio of predictions changed with noises and Confusion Matrix for Robustness
(CMR) that represents the percentage of a predicted class without noise being
predicted to different classes under noise. Besides, we repeatedly trained the
model for 100 times using exactly the same training procedure, hyper-parameter
settings, as well as the training and validation dataset to investigate
robustness of models due to inherent randomness in the training process.
Finally, to improve model robustness, we employed an adaptive training scheme
and demonstrated its effectiveness. Experimental and numerical studies
demonstrated robustness issue of a DNN-based multi-class classification model.
The discoveries highlight the needs for evaluating and ensuring DNN model
robustness.
