  Recent advances in weakly supervised text classification mostly focus on
designing sophisticated methods to turn high-level human heuristics into
quality pseudo-labels. In this paper, we revisit the seed matching-based
method, which is arguably the simplest way to generate pseudo-labels, and show
that its power was greatly underestimated. We show that the limited performance
of seed matching is largely due to the label bias injected by the simple
seed-match rule, which prevents the classifier from learning reliable
confidence for selecting high-quality pseudo-labels. Interestingly, simply
deleting the seed words present in the matched input texts can mitigate the
label bias and help learn better confidence. Subsequently, the performance
achieved by seed matching can be improved significantly, making it on par with
or even better than the state-of-the-art. Furthermore, to handle the case when
the seed words are not made known, we propose to simply delete the word tokens
in the input text randomly with a high deletion ratio. Remarkably, seed
matching equipped with this random deletion method can often achieve even
better performance than that with seed deletion.
