  In this paper, we studied the federated bilevel optimization problem, which
has widespread applications in machine learning. In particular, we developed
two momentum-based algorithms for optimizing this kind of problem and
established the convergence rate of our two algorithms, providing the sample
and communication complexities. Importantly, to the best of our knowledge, our
convergence rate is the first one achieving the linear speedup with respect to
the number of devices for federated bilevel optimization algorithms. At last,
our extensive experimental results confirm the effectiveness of our two
algorithms.
