  Convolutional neural networks (CNNs) have gained increasing popularity and
versatility in recent decades, finding applications in diverse domains. These
remarkable achievements are greatly attributed to the support of extensive
datasets with precise labels. However, annotating image datasets is intricate
and complex, particularly in the case of multi-label datasets. Hence, the
concept of partial-label setting has been proposed to reduce annotation costs,
and numerous corresponding solutions have been introduced. The evaluation
methods for these existing solutions have been primarily based on accuracy.
That is, their performance is assessed by their predictive accuracy on the test
set. However, we insist that such an evaluation is insufficient and one-sided.
On one hand, since the quality of the test set has not been evaluated, the
assessment results are unreliable. On the other hand, the partial-label problem
may also be raised by undergoing adversarial attacks. Therefore, incorporating
robustness into the evaluation system is crucial. For this purpose, we first
propose two attack models to generate multiple partial-label datasets with
varying degrees of label missing rates. Subsequently, we introduce a
lightweight partial-label solution using pseudo-labeling techniques and a
designed loss function. Then, we employ D-Score to analyze both the proposed
and existing methods to determine whether they can enhance robustness while
improving accuracy. Extensive experimental results demonstrate that while
certain methods may improve accuracy, the enhancement in robustness is not
significant, and in some cases, it even diminishes.
