  Current clustering-based Open Relation Extraction (OpenRE) methods usually
adopt a two-stage pipeline. The first stage simultaneously learns relation
representations and assignments. The second stage manually labels several
instances and thus names the relation for each cluster. However, unsupervised
objectives struggle to optimize the model to derive accurate clustering
assignments, and the number of clusters has to be supplied in advance. In this
paper, we present a novel setting, named actively supervised clustering for
OpenRE. Our insight lies in that clustering learning and relation labeling can
be alternately performed, providing the necessary guidance for clustering
without a significant increase in human effort. The key to the setting is
selecting which instances to label. Instead of using classical active labeling
strategies designed for fixed known classes, we propose a new strategy, which
is applicable to dynamically discover clusters of unknown relations.
Experimental results show that our method is able to discover almost all
relational clusters in the data and improve the SOTA methods by 10.3\% and
5.2\%, on two datasets respectively.
