  Clustering of publication networks is an efficient way to obtain
classifications of large collections of research publications. Such
classifications can be used to, e.g., detect research topics, normalize
citation relations, or explore the publication output of a unit. Citation
networks can be created using a variety of approaches. Best practices to obtain
classifications using clustering have been investigated, in particular the
performance of different publication-publication relatedness measures. However,
evaluation of different approaches to normalization of citation relations have
not been explored to the same extent. In this paper, we evaluate five
approaches to normalization of direct citation relations with respect to
clustering solution quality in four data sets. A sixth approach is evaluated
using no normalization. To assess the quality of clustering solutions, we use
three measures. (1) We compare the clustering solution to the reference lists
of a set of publications using the Adjusted Rand Index. (2) Using the Sihouette
width measure, we quantity to which extent the publications have relations to
other clusters than the one they have been assigned to. (3) We propose a
measure that captures publications that have probably been inaccurately
assigned. The results clearly show that normalization is preferred over
unnormalized direct citation relations. Furthermore, the results indicate that
the fractional normalization approach, which can be considered the standard
approach, causes inaccurate assignments. The geometric normalization approach
has a similar performance as the fractional approach regarding Adjusted Rand
Index and Silhouette width but leads to fewer inaccurate assignments. We
therefore believe that the geometric approach may be preferred over the
fractional approach.
