  There is a great diversity of clustering and community detection algorithms,
which are key components of many data analysis and exploration systems. To the
best of our knowledge, however, there does not exist yet any uniform
benchmarking framework, which is publicly available and suitable for the
parallel benchmarking of diverse clustering algorithms on a wide range of
synthetic and real-world datasets. In this paper, we introduce Clubmark, a new
extensible framework that aims to fill this gap by providing a parallel
isolation benchmarking platform for clustering algorithms and their evaluation
on NUMA servers. Clubmark allows for fine-grained control over various
execution variables (timeouts, memory consumption, CPU affinity and cache
policy) and supports the evaluation of a wide range of clustering algorithms
including multi-level, hierarchical and overlapping clustering techniques on
both weighted and unweighted input networks with built-in evaluation of several
extrinsic and intrinsic measures. Our framework is open-source and provides a
consistent and systematic way to execute, evaluate and profile clustering
techniques considering a number of aspects that are often missing in
state-of-the-art frameworks and benchmarking systems.
