  Data quality is critical for multimedia tasks, while various types of
systematic flaws are found in image benchmark datasets, as discussed in recent
work. In particular, the existence of the semantic gap problem leads to a
many-to-many mapping between the information extracted from an image and its
linguistic description. This unavoidable bias further leads to poor performance
on current computer vision tasks. To address this issue, we introduce a
Knowledge Representation (KR)-based methodology to provide guidelines driving
the labeling process, thereby indirectly introducing intended semantics in ML
models. Specifically, an iterative refinement-based annotation method is
proposed to optimize data labeling by organizing objects in a classification
hierarchy according to their visual properties, ensuring that they are aligned
with their linguistic descriptions. Preliminary results verify the
effectiveness of the proposed method.
