  Most network embedding algorithms consist in measuring co-occurrences of
nodes via random walks then learning the embeddings using Skip-Gram with
Negative Sampling. While it has proven to be a relevant choice, there are
alternatives, such as GloVe, which has not been investigated yet for network
embedding. Even though SGNS better handles non co-occurrence than GloVe, it has
a worse time-complexity. In this paper, we propose a matrix factorization
approach for network embedding, inspired by GloVe, that better handles non
co-occurrence with a competitive time-complexity. We also show how to extend
this model to deal with networks where nodes are documents, by simultaneously
learning word, node and document representations. Quantitative evaluations show
that our model achieves state-of-the-art performance, while not being so
sensitive to the choice of hyper-parameters. Qualitatively speaking, we show
how our model helps exploring a network of documents by generating
complementary network-oriented and content-oriented keywords.
