  Recent research demonstrates that external knowledge injection can advance
pre-trained language models (PLMs) in a variety of downstream NLP tasks.
However, existing knowledge injection methods are either applicable to
structured knowledge or unstructured knowledge, lacking a unified usage. In
this paper, we propose a UNified knowledge inTERface, UNTER, to provide a
unified perspective to exploit both structured knowledge and unstructured
knowledge. In UNTER, we adopt the decoder as a unified knowledge interface,
aligning span representations obtained from the encoder with their
corresponding knowledge. This approach enables the encoder to uniformly invoke
span-related knowledge from its parameters for downstream applications.
Experimental results show that, with both forms of knowledge injected, UNTER
gains continuous improvements on a series of knowledge-driven NLP tasks,
including entity typing, named entity recognition and relation extraction,
especially in low-resource scenarios.
