  Utilizing pre-trained language models has achieved great success for neural
document ranking. Limited by the computational and memory requirements, long
document modeling becomes a critical issue. Recent works propose to modify the
full attention matrix in Transformer by designing sparse attention patterns.
However, most of them only focus on local connections of terms within a
fixed-size window. How to build suitable remote connections between terms to
better model document representation remains underexplored. In this paper, we
propose the model Socialformer, which introduces the characteristics of social
networks into designing sparse attention patterns for long document modeling in
document ranking. Specifically, we consider several attention patterns to
construct a graph like social networks. Endowed with the characteristic of
social networks, most pairs of nodes in such a graph can reach with a short
path while ensuring the sparsity. To facilitate efficient calculation, we
segment the graph into multiple subgraphs to simulate friend circles in social
scenarios. Experimental results confirm the effectiveness of our model on long
document modeling.
