  Nowadays, how to effectively evaluate visual properties has become a popular
topic for fine-grained visual comprehension. In this paper we study the problem
of how to estimate such visual properties from a ranking perspective with the
help of the annotators from online crowdsourcing platforms. The main challenges
of our task are two-fold. On one hand, the annotations often contain
contaminated information, where a small fraction of label flips might ruin the
global ranking of the whole dataset. On the other hand, considering the large
data capacity, the annotations are often far from being complete. What is
worse, there might even exist imbalanced annotations where a small subset of
samples are frequently annotated. Facing such challenges, we propose a robust
ranking framework based on the principle of Hodge decomposition of imbalanced
and incomplete ranking data. According to the HodgeRank theory, we find that
the major source of the contamination comes from the cyclic ranking component
of the Hodge decomposition. This leads us to an outlier detection formulation
as sparse approximations of the cyclic ranking projection. Taking a step
further, it facilitates a novel outlier detection model as Huber's LASSO in
robust statistics. Moreover, simple yet scalable algorithms are developed based
on Linearized Bregman Iteration to achieve an even less biased estimator.
Statistical consistency of outlier detection is established in both cases under
nearly the same conditions. Our studies are supported by experiments with both
simulated examples and real-world data. The proposed framework provides us a
promising tool for robust ranking with large scale crowdsourcing data arising
from computer vision.
