  The use of bag of visual words (BOW) model for modelling images based on
local invariant features computed at interest point locations has become a
standard choice for many computer vision tasks. Visual vocabularies generated
from image feature vectors are expected to produce visual words that are
discriminative to improve the performance of image annotation systems. Most
techniques that adopt the BOW model in annotating images declined favorable
information that can be mined from image categories to build discriminative
visual vocabularies. To this end, this paper introduces a detailed framework
for automatically annotating natural scene images with local semantic labels
from a predefined vocabulary. The framework is based on a hypothesis that
assumes that, in natural scenes, intermediate semantic concepts are correlated
with the local keypoints. Based on this hypothesis, image regions can be
efficiently represented by BOW model and using a machine learning approach,
such as SVM, to label image regions with semantic annotations. Another
objective of this paper is to address the implications of generating visual
vocabularies from image halves, instead of producing them from the whole image,
on the performance of annotating image regions with semantic labels. All
BOW-based approaches as well as baseline methods have been extensively
evaluated on 6-categories dataset of natural scenes using the SVM and KNN
classifiers. The reported results have shown the plausibility of using the BOW
model to represent the semantic information of image regions and thus to
automatically annotate image regions with labels.
