  The availability of large annotated data can be a critical bottleneck in
training machine learning algorithms successfully, especially when applied to
diverse domains. Weak supervision offers a promising alternative by
accelerating the creation of labeled training data using domain-specific rules.
However, it requires users to write a diverse set of high-quality rules to
assign labels to the unlabeled data. Automatic Rule Induction (ARI) approaches
circumvent this problem by automatically creating rules from features on a
small labeled set and filtering a final set of rules from them. In the ARI
approach, the crucial step is to filter out a set of a high-quality useful
subset of rules from the large set of automatically created rules. In this
paper, we propose an algorithm (Filtering of Automatically Induced Rules) to
filter rules from a large number of automatically induced rules using
submodular objective functions that account for the collective precision,
coverage, and conflicts of the rule set. We experiment with three ARI
approaches and five text classification datasets to validate the superior
performance of our algorithm with respect to several semi-supervised label
aggregation approaches. Further, we show that achieves statistically
significant results in comparison to existing rule-filtering approaches.
