  Benchmarking the performance of community detection methods on empirical
social network data has been identified as critical for improving these
methods. In particular, while most current research focuses on detecting
communities in data that has been digitally extracted from large social media
and telecommunications services, most evaluation of this research is based on
small, hand-curated datasets. We argue that these two types of networks differ
so significantly that by evaluating algorithms solely on the former, we know
little about how well they perform on the latter. To address this problem, we
consider the difficulties that arise in constructing benchmarks based on
digitally extracted network data, and propose a task-based strategy which we
feel addresses these difficulties. To demonstrate that our scheme is effective,
we use it to carry out a substantial benchmark based on Facebook data. The
benchmark reveals that some of the most popular algorithms fail to detect
fine-grained community structure.
