  Federated learning is a distributed collaborative machine learning paradigm
that has gained strong momentum in recent years. In federated learning, a
central server periodically coordinates models with clients and aggregates the
models trained locally by clients without necessitating access to local data.
Despite its potential, the implementation of federated learning continues to
encounter several challenges, predominantly the slow convergence that is
largely due to data heterogeneity. The slow convergence becomes particularly
problematic in cross-device federated learning scenarios where clients may be
strongly limited by computing power and storage space, and hence counteracting
methods that induce additional computation or memory cost on the client side
such as auxiliary objective terms and larger training iterations can be
impractical. In this paper, we propose a novel federated aggregation strategy,
TurboSVM-FL, that poses no additional computation burden on the client side and
can significantly accelerate convergence for federated classification task,
especially when clients are "lazy" and train their models solely for few epochs
for next global aggregation. TurboSVM-FL extensively utilizes support vector
machine to conduct selective aggregation and max-margin spread-out
regularization on class embeddings. We evaluate TurboSVM-FL on multiple
datasets including FEMNIST, CelebA, and Shakespeare using user-independent
validation with non-iid data distribution. Our results show that TurboSVM-FL
can significantly outperform existing popular algorithms on convergence rate
and reduce communication rounds while delivering better test metrics including
accuracy, F1 score, and MCC.
