  Biomedical entity linking is the task of linking entity mentions in a
biomedical document to referent entities in a knowledge base. Recently, many
BERT-based models have been introduced for the task. While these models have
achieved competitive results on many datasets, they are computationally
expensive and contain about 110M parameters. Little is known about the factors
contributing to their impressive performance and whether the
over-parameterization is needed. In this work, we shed some light on the inner
working mechanisms of these large BERT-based models. Through a set of probing
experiments, we have found that the entity linking performance only changes
slightly when the input word order is shuffled or when the attention scope is
limited to a fixed window size. From these observations, we propose an
efficient convolutional neural network with residual connections for biomedical
entity linking. Because of the sparse connectivity and weight sharing
properties, our model has a small number of parameters and is highly efficient.
On five public datasets, our model achieves comparable or even better linking
accuracy than the state-of-the-art BERT-based models while having about 60
times fewer parameters.
