  We propose an unsupervised method to extract keywords and keyphrases from
texts based on a pre-trained language model (LM) and Shannon's information
maximization. Specifically, our method extracts phrases having the highest
conditional entropy under the LM. The resulting set of keyphrases turns out to
solve a relevant information-theoretic problem: if provided as side
information, it leads to the expected minimal binary code length in compressing
the text using the LM and an entropy encoder. Alternately, the resulting set is
an approximation via a causal LM to the set of phrases that minimize the
entropy of the text when conditioned upon it. Empirically, the method provides
results comparable to the most commonly used methods in various keyphrase
extraction benchmark challenges.
