  In this paper, we explore text classification with extremely weak
supervision, i.e., only relying on the surface text of class names. This is a
more challenging setting than the seed-driven weak supervision, which allows a
few seed words per class. We opt to attack this problem from a representation
learning perspective -- ideal document representations should lead to nearly
the same results between clustering and the desired classification. In
particular, one can classify the same corpus differently (e.g., based on topics
and locations), so document representations should be adaptive to the given
class names. We propose a novel framework X-Class to realize the adaptive
representations. Specifically, we first estimate class representations by
incrementally adding the most similar word to each class until inconsistency
arises. Following a tailored mixture of class attention mechanisms, we obtain
the document representation via a weighted average of contextualized word
representations. With the prior of each document assigned to its nearest class,
we then cluster and align the documents to classes. Finally, we pick the most
confident documents from each cluster to train a text classifier. Extensive
experiments demonstrate that X-Class can rival and even outperform seed-driven
weakly supervised methods on 7 benchmark datasets. Our dataset and code are
released at https://github.com/ZihanWangKi/XClass/ .
