  There has been appreciable progress in unsupervised network representation
learning (UNRL) approaches over graphs recently with flexible random-walk
approaches, new optimization objectives and deep architectures. However, there
is no common ground for systematic comparison of embeddings to understand their
behavior for different graphs and tasks. In this paper we theoretically group
different approaches under a unifying framework and empirically investigate the
effectiveness of different network representation methods. In particular, we
argue that most of the UNRL approaches either explicitly or implicit model and
exploit context information of a node. Consequently, we propose a framework
that casts a variety of approaches -- random walk based, matrix factorization
and deep learning based -- into a unified context-based optimization function.
We systematically group the methods based on their similarities and
differences. We study the differences among these methods in detail which we
later use to explain their performance differences (on downstream tasks). We
conduct a large-scale empirical study considering 9 popular and recent UNRL
techniques and 11 real-world datasets with varying structural properties and
two common tasks -- node classification and link prediction. We find that there
is no single method that is a clear winner and that the choice of a suitable
method is dictated by certain properties of the embedding methods, task and
structural properties of the underlying graph. In addition we also report the
common pitfalls in evaluation of UNRL methods and come up with suggestions for
experimental design and interpretation of results.
