  Acronym disambiguation means finding the correct meaning of an ambiguous
acronym from the dictionary in a given sentence, which is one of the key points
for scientific document understanding (SDU@AAAI-22). Recently, many attempts
have tried to solve this problem via fine-tuning the pre-trained masked
language models (MLMs) in order to obtain a better acronym representation.
However, the acronym meaning is varied under different contexts, whose
corresponding phrase representation mapped in different directions lacks
discrimination in the entire vector space. Thus, the original representations
of the pre-trained MLMs are not ideal for the acronym disambiguation task. In
this paper, we propose a Simple framework for Contrastive Learning of Acronym
Disambiguation (SimCLAD) method to better understand the acronym meanings.
Specifically, we design a continual contrastive pre-training method that
enhances the pre-trained model's generalization ability by learning the
phrase-level contrastive distributions between true meaning and ambiguous
phrases. The results on the acronym disambiguation of the scientific domain in
English show that the proposed method outperforms all other competitive
state-of-the-art (SOTA) methods.
