  Many algorithms have been proposed in the last ten years for the discovery of
dynamic communities. However, these methods are seldom compared between
themselves. In this article, we propose a generator of dynamic graphs with
planted evolving community structure, as a benchmark to compare and evaluate
such algorithms. Unlike previously proposed benchmarks, it is able to specify
any desired evolving community structure through a descriptive language, and
then to generate the corresponding progressively evolving network. We
empirically evaluate six existing algorithms for dynamic community detection in
terms of instantaneous and longitudinal similarity with the planted ground
truth, smoothness of dynamic partitions, and scalability. We notably observe
different types of weaknesses depending on their approach to ensure smoothness,
namely Glitches, Oversimplification and Identity loss. Although no method
arises as a clear winner, we observe clear differences between methods, and we
identified the fastest, those yielding the most smoothed or the most accurate
solutions at each step.
