  In this extended abstract, we present an algorithm that learns a similarity
measure between documents from the network topology of a structured corpus. We
leverage the Scaled Dot-Product Attention, a recently proposed attention
mechanism, to design a mutual attention mechanism between pairs of documents.
To train its parameters, we use the network links as supervision. We provide
preliminary experiment results with a citation dataset on two prediction tasks,
demonstrating the capacity of our model to learn a meaningful textual
similarity.
