  Salient object detection is evaluated using binary ground truth with the
labels being salient object class and background. In this paper, we corroborate
based on three subjective experiments on a novel image dataset that objects in
natural images are inherently perceived to have varying levels of importance.
Our dataset, named SalMoN (saliency in multi-object natural images), has 588
images containing multiple objects. The subjective experiments performed record
spontaneous attention and perception through eye fixation duration, point
clicking and rectangle drawing. As object saliency in a multi-object image is
inherently multi-level, we propose that salient object detection must be
evaluated for the capability to detect all multi-level salient objects apart
from the salient object class detection capability. For this purpose, we
generate multi-level maps as ground truth corresponding to all the dataset
images using the results of the subjective experiments, with the labels being
multi-level salient objects and background. We then propose the use of mean
absolute error, Kendall's rank correlation and average area under
precision-recall curve to evaluate existing salient object detection methods on
our multi-level saliency ground truth dataset. Approaches that represent
saliency detection on images as local-global hierarchical processing of a graph
perform well in our dataset.
