  In this paper we take a problem of unsupervised nodes clustering on graphs
and show how recent advances in attention models can be applied successfully in
a "hard" regime of the problem. We propose an unsupervised algorithm that
encodes Bethe Hessian embeddings by optimizing soft modularity loss and argue
that our model is competitive to both classical and Graph Neural Network (GNN)
models while it can be trained on a single graph.
