  Labeling data via rules-of-thumb and minimal label supervision is central to
Weak Supervision, a paradigm subsuming subareas of machine learning such as
crowdsourced learning and semi-supervised ensemble learning. By using this
labeled data to train modern machine learning methods, the cost of acquiring
large amounts of hand labeled data can be ameliorated. Approaches to combining
the rules-of-thumb falls into two camps, reflecting different ideologies of
statistical estimation. The most common approach, exemplified by the
Dawid-Skene model, is based on probabilistic modeling. The other, developed in
the work of Balsubramani-Freund and others, is adversarial and game-theoretic.
We provide a variety of statistical results for the adversarial approach under
log-loss: we characterize the form of the solution, relate it to logistic
regression, demonstrate consistency, and give rates of convergence. On the
other hand, we find that probabilistic approaches for the same model class can
fail to be consistent. Experimental results are provided to corroborate the
theoretical results.
