  Contrastive Language-Image Pretraining (CLIP) efficiently learns visual
concepts by pre-training with natural language supervision. CLIP and its visual
encoder have been explored on various vision and language tasks and achieve
strong zero-shot or transfer learning performance. However, the application of
its text encoder solely for text understanding has been less explored. In this
paper, we find that the text encoder of CLIP actually demonstrates strong
ability for phrase understanding, and can even significantly outperform popular
language models such as BERT with a properly designed prompt. Extensive
experiments validate the effectiveness of our method across different datasets
and domains on entity clustering and entity set expansion tasks.
