  Federated learning is a new machine learning paradigm. The goal is to build a
machine learning model from the data sets distributed on multiple devices
so-called an isolated data island, while keeping their data secure and private.
Most existing federated learning benchmarks work manually splits commonly used
public datasets into partitions to simulate real world isolated data island
scenarios. Still, this simulation fails to capture real world isolated data
island intrinsic characteristics. This paper presents a federated learning (FL)
benchmark suite named FLBench. FLBench contains three domains: medical,
financial, and AIoT. By configuring various domains, FLBench is qualified to
evaluate federated learning systems and algorithms essential aspects, like
communication, scenario transformation, privacy-preserving, data distribution
heterogeneity, and cooperation strategy. Hence, it becomes a promising platform
for developing novel federated learning algorithms. Currently, FLBench is open
sourced and in fast evolution. We package it as an automated deployment tool.
The benchmark suite is available from
https://www.benchcouncil.org/flbench.html.
