  Salient object detection is a problem that has been considered in detail and
\textcolor{black}{many solutions have been proposed}. In this paper, we argue
that work to date has addressed a problem that is relatively ill-posed.
Specifically, there is not universal agreement about what constitutes a salient
object when multiple observers are queried. This implies that some objects are
more likely to be judged salient than others, and implies a relative rank
exists on salient objects. Initially, we present a novel deep learning solution
based on a hierarchical representation of relative saliency and stage-wise
refinement. Further to this, we present data, analysis and baseline benchmark
results towards addressing the problem of salient object ranking. Methods for
deriving suitable ranked salient object instances are presented, along with
metrics suitable to measuring algorithm performance. In addition, we show how a
derived dataset can be successively refined to provide cleaned results that
correlate well with pristine ground truth in its characteristics and value for
training and testing models. Finally, we provide a comparison among prevailing
algorithms that address salient object ranking or detection to establish
initial baselines providing a basis for comparison with future efforts
addressing this problem. \textcolor{black}{The source code and data are
publicly available via our project page:}
\textrm{\href{https://ryersonvisionlab.github.io/cocosalrank.html}{ryersonvisionlab.github.io/cocosalrank}}
