  Federated learning has emerged as an umbrella term for centralized
coordination strategies in multi-agent environments. While many federated
learning architectures process data in an online manner, and are hence adaptive
by nature, most performance analyses assume static optimization problems and
offer no guarantees in the presence of drifts in the problem solution or data
characteristics. We consider a federated learning model where at every
iteration, a random subset of available agents perform local updates based on
their data. Under a non-stationary random walk model on the true minimizer for
the aggregate optimization problem, we establish that the performance of the
architecture is determined by three factors, namely, the data variability at
each agent, the model variability across all agents, and a tracking term that
is inversely proportional to the learning rate of the algorithm. The results
clarify the trade-off between convergence and tracking performance.
