  Federated Learning (FL) is an emerging direction in distributed machine
learning (ML) that enables in-situ model training and testing on edge data.
Despite having the same end goals as traditional ML, FL executions differ
significantly in scale, spanning thousands to millions of participating
devices. As a result, data characteristics and device capabilities vary widely
across clients. Yet, existing efforts randomly select FL participants, which
leads to poor model and system efficiency.
  In this paper, we propose Oort to improve the performance of federated
training and testing with guided participant selection. With an aim to improve
time-to-accuracy performance in model training, Oort prioritizes the use of
those clients who have both data that offers the greatest utility in improving
model accuracy and the capability to run training quickly. To enable FL
developers to interpret their results in model testing, Oort enforces their
requirements on the distribution of participant data while improving the
duration of federated testing by cherry-picking clients. Our evaluation shows
that, compared to existing participant selection mechanisms, Oort improves
time-to-accuracy performance by 1.2x-14.1x and final model accuracy by
1.3%-9.8%, while efficiently enforcing developer-specified model testing
criteria at the scale of millions of clients.
