  Nowadays, billions of phones, IoT and edge devices around the world generate
data continuously, enabling many Machine Learning (ML)-based products and
applications. However, due to increasing privacy concerns and regulations,
these data tend to reside on devices (clients) instead of being centralized for
performing traditional ML model training. Federated Learning (FL) is a
distributed approach in which a single server and multiple clients
collaboratively build an ML model without moving data away from clients.
Whereas existing studies on FL have their own experimental evaluations, most
experiments were conducted using a simulation setting or a small-scale testbed.
This might limit the understanding of FL implementation in realistic
environments. In this empirical study, we systematically conduct extensive
experiments on a large network of IoT and edge devices (called IoT-Edge
devices) to present FL real-world characteristics, including learning
performance and operation (computation and communication) costs. Moreover, we
mainly concentrate on heterogeneous scenarios, which is the most challenging
issue of FL. By investigating the feasibility of on-device implementation, our
study provides valuable insights for researchers and practitioners, promoting
the practicality of FL and assisting in improving the current design of real FL
systems.
