  Complex structures commonly exist in natural images. When an image contains
small-scale high-contrast patterns either in the background or foreground,
saliency detection could be adversely affected, resulting erroneous and
non-uniform saliency assignment. The issue forms a fundamental challenge for
prior methods. We tackle it from a scale point of view and propose a
multi-layer approach to analyze saliency cues. Different from varying patch
sizes or downsizing images, we measure region-based scales. The final saliency
values are inferred optimally combining all the saliency cues in different
scales using hierarchical inference. Through our inference model, single-scale
information is selected to obtain a saliency map. Our method improves detection
quality on many images that cannot be handled well traditionally. We also
construct an extended Complex Scene Saliency Dataset (ECSSD) to include complex
but general natural images.
