  With the booming of pre-trained transformers, representation-based models
based on Siamese transformer encoders have become mainstream techniques for
efficient text matching. However, these models suffer from severe performance
degradation due to the lack of interaction between the text pair, compared with
interaction-based models. Prior arts attempt to address this through performing
extra interaction for Siamese encoded representations, while the interaction
during encoding is still ignored. To remedy this, we propose a \textit{Virtual}
InteRacTion mechanism (VIRT) to transfer interactive knowledge from
interaction-based models into Siamese encoders through attention map
distillation. As a train-time-only component, VIRT could completely maintain
the high efficiency of the Siamese structure and brings no extra computation
cost during inference. To fully utilize the learned interactive knowledge, we
further design a VIRT-adapted interaction strategy. Experimental results on
multiple text matching datasets demonstrate that our method outperforms
state-of-the-art representation-based models. What's more, VIRT can be easily
integrated into existing representation-based methods to achieve further
improvements.
