  Most conventional Federated Learning (FL) models are using a star network
topology where all users aggregate their local models at a single server (e.g.,
a cloud server). That causes significant overhead in terms of both
communications and computing at the server, delaying the training process,
especially for large scale FL systems with straggling nodes. This paper
proposes a novel edge network architecture that enables decentralizing the
model aggregation process at the server, thereby significantly reducing the
training delay for the whole FL network. Specifically, we design a
highly-effective in-network computation protocol (INC) consisting of a user
scheduling mechanism, an in-network aggregation process (INA) which is designed
for both primal- and primal-dual methods in distributed machine learning
problems, and a network routing algorithm. Under the proposed INA, we then
formulate a joint routing and resource optimization problem, aiming to minimize
the aggregation latency. The problem is NP-hard, and thus we propose a
polynomial time routing algorithm which can achieve near optimal performance
with a theoretical bound. Simulation results showed that the proposed INC
framework can not only help reduce the FL training latency, up to 5.6 times,
but also significantly decrease cloud's traffic and computing overhead. This
can enable large-scale FL.
