  Worker quality control is a crucial aspect of crowdsourcing systems;
typically occupying a large fraction of the time and money invested on
crowdsourcing. In this work, we devise techniques to generate confidence
intervals for worker error rate estimates, thereby enabling a better evaluation
of worker quality. We show that our techniques generate correct confidence
intervals on a range of real-world datasets, and demonstrate wide applicability
by using them to evict poorly performing workers, and provide confidence
intervals on the accuracy of the answers.
