  Current medical image classification efforts mainly aim for higher average
performance, often neglecting the balance between different classes. This can
lead to significant differences in recognition accuracy between classes and
obvious recognition weaknesses. Without the support of massive data, deep
learning faces challenges in fine-grained classification of fatty liver. In
this paper, we propose an innovative deep learning framework that combines
feature decoupling and adaptive adversarial training. Firstly, we employ two
iteratively compressed decouplers to supervised decouple common features and
specific features related to fatty liver in abdominal ultrasound images.
Subsequently, the decoupled features are concatenated with the original image
after transforming the color space and are fed into the classifier. During
adversarial training, we adaptively adjust the perturbation and balance the
adversarial strength by the accuracy of each class. The model will eliminate
recognition weaknesses by correctly classifying adversarial samples, thus
improving recognition robustness. Finally, the accuracy of our method improved
by 4.16%, achieving 82.95%. As demonstrated by extensive experiments, our
method is a generalized learning framework that can be directly used to
eliminate the recognition weaknesses of any classifier while improving its
average performance. Code is available at https://github.com/HP-ML/MICCAI2024.
