  The rapid development of parallel and distributed computing paradigms has
brought about great revolution in computing. Thanks to the intrinsic
parallelism of evolutionary computation (EC), it is natural to implement EC on
parallel and distributed computing systems. On the one hand, the computing
power provided by parallel computing systems can significantly improve the
efficiency and scalability of EC. On the other hand, data are collected and
processed in a distributed manner, which brings a novel development direction
and new challenges to EC. In this paper, we intend to give a systematic review
on distributed EC (DEC). First, a new taxonomy for DEC is proposed from top
design mechanism to bottom implementation mechanism. Based on this taxonomy,
existing studies on DEC are reviewed in terms of purpose, parallel structure of
the algorithm, parallel model for implementation, and the implementation
environment. Second, we clarify two major purposes of DEC, i.e., improving
efficiency through parallel processing for centralized optimization and
cooperating distributed individuals/sub-populations with partial information to
perform distributed optimization. Third, noting that the latter purpose of DEC
is an emerging and attractive trend for EC with the booming of spatially
distributed paradigms, this paper gives a systematic definition of the
distributed optimization and classifies it into dimension distributed-, data
distributed-, and objective distributed-optimization problems. Formal
formulations for these problems are provided and various DEC studies on these
problems are reviewed. We also discuss challenges and potential research
directions, aiming to enlighten the design of DEC and pave the way for future
developments.
