  Federated Learning (FL) has become a key choice for distributed machine
learning. Initially focused on centralized aggregation, recent works in FL have
emphasized greater decentralization to adapt to the highly heterogeneous
network edge. Among these, Hierarchical, Device-to-Device and Gossip Federated
Learning (HFL, D2DFL \& GFL respectively) can be considered as foundational FL
algorithms employing fundamental aggregation strategies. A number of FL
algorithms were subsequently proposed employing multiple fundamental
aggregation schemes jointly. Existing research, however, subjects the FL
algorithms to varied conditions and gauges the performance of these algorithms
mainly against Federated Averaging (FedAvg) only. This work consolidates the FL
landscape and offers an objective analysis of the major FL algorithms through a
comprehensive cross-evaluation for a wide range of operating conditions. In
addition to the three foundational FL algorithms, this work also analyzes six
derived algorithms. To enable a uniform assessment, a multi-FL framework named
FLAGS: Federated Learning AlGorithms Simulation has been developed for rapid
configuration of multiple FL algorithms. Our experiments indicate that fully
decentralized FL algorithms achieve comparable accuracy under multiple
operating conditions, including asynchronous aggregation and the presence of
stragglers. Furthermore, decentralized FL can also operate in noisy
environments and with a comparably higher local update rate. However, the
impact of extremely skewed data distributions on decentralized FL is much more
adverse than on centralized variants. The results indicate that it may not be
necessary to restrict the devices to a single FL algorithm; rather, multi-FL
nodes may operate with greater efficiency.
