  Modern machine learning is migrating to the era of complex models, which
requires a plethora of well-annotated data. While crowdsourcing is a promising
tool to achieve this goal, existing crowdsourcing approaches barely acquire a
sufficient amount of high-quality labels. In this paper, motivated by the
"Guess-with-Hints" answer strategy from the Millionaire game show, we introduce
the hint-guided approach into crowdsourcing to deal with this challenge. Our
approach encourages workers to get help from hints when they are unsure of
questions. Specifically, we propose a hybrid-stage setting, consisting of the
main stage and the hint stage. When workers face any uncertain question on the
main stage, they are allowed to enter the hint stage and look up hints before
making any answer. A unique payment mechanism that meets two important design
principles for crowdsourcing is developed. Besides, the proposed mechanism
further encourages high-quality workers less using hints, which helps identify
and assigns larger possible payment to them. Experiments are performed on
Amazon Mechanical Turk, which show that our approach ensures a sufficient
number of high-quality labels with low expenditure and detects high-quality
workers.
