  Node embedding methods find latent lower-dimensional representations which
are used as features in machine learning models. In the last few years, these
methods have become extremely popular as a replacement for manual feature
engineering. Since authors use various approaches for the evaluation of node
embedding methods, existing studies can rarely be efficiently and accurately
compared. We address this issue by developing a process for a fair and
objective evaluation of node embedding procedures w.r.t. node classification.
This process supports researchers and practitioners to compare new and existing
methods in a reproducible way. We apply this process to four popular node
embedding methods and make valuable observations. With an appropriate
combination of hyperparameters, good performance can be achieved even with
embeddings of lower dimensions, which is positive for the run times of the
downstream machine learning task and the embedding algorithm. Multiple
hyperparameter combinations yield similar performance. Thus, no extensive,
time-consuming search is required to achieve reasonable performance in most
cases.
