  This paper studies the problem of injecting factual knowledge into large
pre-trained language models. We train adapter modules on parts of the
ConceptNet knowledge graph using the masked language modeling objective and
evaluate the success of the method by a series of probing experiments on the
LAMA probe. Mean P@K curves for different configurations indicate that the
technique is effective, increasing the performance on subsets of the LAMA probe
for large values of k by adding as little as 2.1% additional parameters to the
original models.
