  Background. Clustering analysis discovers hidden structures in a data set by
partitioning them into disjoint clusters. Robust accuracy measures that
evaluate the goodness of clustering results are critical for algorithm
development and model diagnosis. Common problems of current clustering accuracy
measures include overlooking unmatched clusters, biases towards excessive
clusters, unstable baselines, and difficult interpretation. In this study, we
presented a novel accuracy measure, J-score, that addresses these issues.
  Methods. Given a data set with known class labels, J-score quantifies how
well the hypothetical clusters produced by clustering analysis recover the true
classes. It starts with bidirectional set matching to identify the
correspondence between true classes and hypothetical clusters based on Jaccard
index. It then computes two weighted sums of Jaccard indices measuring the
reconciliation from classes to clusters and vice versa. The final J-score is
the harmonic mean of the two weighted sums.
  Results. Via simulation studies, we evaluated the performance of J-score and
compared with existing measures. Our results show that J-score is effective in
distinguishing partition structures that differ only by unmatched clusters,
rewarding correct inference of class numbers, addressing biases towards
excessive clusters, and having a relatively stable baseline. The simplicity of
its calculation makes the interpretation straightforward. It is a valuable tool
complementary to other accuracy measures. We released an R/jScore package
implementing the algorithm.
