  Recent work has shown that 8-bit floating point (FP8) can be used for
efficiently training neural networks with reduced computational overhead
compared to training in FP32/FP16. In this work, we investigate the use of FP8
training in a federated learning context. This brings not only the usual
benefits of FP8 which are desirable for on-device training at the edge, but
also reduces client-server communication costs due to significant weight
compression. We present a novel method for combining FP8 client training while
maintaining a global FP32 server model and provide convergence analysis.
Experiments with various machine learning models and datasets show that our
method consistently yields communication reductions of at least 2.9x across a
variety of tasks and models compared to an FP32 baseline.
