  The semantic gap is defined as the difference between the linguistic
representations of the same concept, which usually leads to misunderstanding
between individuals with different knowledge backgrounds. Since linguistically
annotated images are extensively used for training machine learning models,
semantic gap problem (SGP) also results in inevitable bias on image annotations
and further leads to poor performance on current computer vision tasks. To
address this problem, we propose a novel unsupervised method to build visual
semantics aware object hierarchy, aiming to get a classification model by
learning from pure-visual information and to dissipate the bias of linguistic
representations caused by SGP. Our intuition in this paper comes from
real-world knowledge representation where concepts are hierarchically
organized, and each concept can be described by a set of features rather than a
linguistic annotation, namely visual semantic. The evaluation consists of two
parts, firstly we apply the constructed hierarchy on the object recognition
task and then we compare our visual hierarchy and existing lexical hierarchies
to show the validity of our method. The preliminary results reveal the
efficiency and potential of our proposed method.
