  In many domains, there are many examples and far fewer labels for those
examples; e.g. we may have access to millions of lines of source code, but
access to only a handful of warnings about that code. In those domains,
semi-supervised learners (SSL) can extrapolate labels from a small number of
examples to the rest of the data. Standard SSL algorithms use ``weak''
knowledge (i.e. those not based on specific SE knowledge) such as (e.g.)
co-train two learners and use good labels from one to train the other. Another
approach of SSL in software analytics is potentially use ``strong'' knowledge
that use SE knowledge. For example, an often-used heuristic in SE is that
unusually large artifacts contain undesired properties (e.g. more bugs). This
paper argues that such ``strong'' algorithms perform better than those
standard, weaker, SSL algorithms. We show this by learning models from labels
generated using weak SSL or our ``stronger'' FRUGAL algorithm. In four domains
(distinguishing security-related bug reports; mitigating bias in
decision-making; predicting issue close time; and (reducing false alarms in
static code warnings), FRUGAL required only 2.5% of the data to be labeled yet
out-performed standard semi-supervised learners that relied on (e.g.) some
domain-independent graph theory concepts. Hence, for future work, we strongly
recommend the use of strong heuristics for semi-supervised learning for SE
applications. To better support other researchers, our scripts and data are
on-line at https://github.com/HuyTu7/FRUGAL.
