  Algorithmic classifications of research publications can be used to study
many different aspects of the science system, such as the organization of
science into fields, the growth of fields, interdisciplinarity, and emerging
topics. How to label the classes in these classifications is a problem that has
not been thoroughly addressed in the literature. In this study we evaluate
different approaches to label the classes in algorithmically constructed
classifications of research publications. We focus on two important choices:
the choice of (1) different bibliographic fields and (2) different approaches
to weight the relevance of terms. To evaluate the different choices, we created
two baselines: one based on the Medical Subject Headings in MEDLINE and another
based on the Science-Metrix journal classification. We tested to what extent
different approaches yield the desired labels for the classes in the two
baselines. Based on our results we recommend extracting terms from titles and
keywords to label classes at high levels of granularity (e.g. topics). At low
levels of granularity (e.g. disciplines) we recommend extracting terms from
journal names and author addresses. We recommend the use of a new approach,
term frequency to specificity ratio, to calculate the relevance of terms.
