  Bi-encoder architectures for distantly-supervised relation extraction are
designed to make use of the complementary information found in text and
knowledge graphs (KG). However, current architectures suffer from two
drawbacks. They either do not allow any sharing between the text encoder and
the KG encoder at all, or, in case of models with KG-to-text attention, only
share information in one direction. Here, we introduce cross-stitch
bi-encoders, which allow full interaction between the text encoder and the KG
encoder via a cross-stitch mechanism. The cross-stitch mechanism allows sharing
and updating representations between the two encoders at any layer, with the
amount of sharing being dynamically controlled via cross-attention-based gates.
Experimental results on two relation extraction benchmarks from two different
domains show that enabling full interaction between the two encoders yields
strong improvements.
