  Finetuning large language models (LLMs) in federated learning (FL) settings
has become important as it allows resource-constrained devices to finetune a
model using private data. However, finetuning LLMs using backpropagation
requires excessive memory (especially from intermediate activations) for
resource-constrained devices. While Forward-mode Auto-Differentiation (AD) can
reduce memory footprint from activations, we observe that directly applying it
to LLM finetuning results in slow convergence and poor accuracy. This work
introduces Spry, an FL algorithm that splits trainable weights of an LLM among
participating clients, such that each client computes gradients using
Forward-mode AD that are closer estimates of the true gradients. Spry achieves
a low memory footprint, high accuracy, and fast convergence. We theoretically
show that the global gradients in Spry are unbiased estimates of true global
gradients for homogeneous data distributions across clients, while
heterogeneity increases bias of the estimates. We also derive Spry's
convergence rate, showing that the gradients decrease inversely proportional to
the number of FL rounds, indicating the convergence up to the limits of
heterogeneity. Empirically, Spry reduces the memory footprint during training
by 1.4-7.1$\times$ in contrast to backpropagation, while reaching comparable
accuracy, across a wide range of language tasks, models, and FL settings. Spry
reduces the convergence time by 1.2-20.3$\times$ and achieves 5.2-13.5\% higher
accuracy against state-of-the-art zero-order methods. When finetuning Llama2-7B
with LoRA, compared to the peak memory usage of 33.9GB of backpropagation, Spry
only consumes 6.2GB of peak memory. For OPT13B, the reduction is from 76.5GB to
10.8GB. Spry makes feasible previously impossible FL deployments on commodity
mobile and edge devices. Source code is available at
https://github.com/Astuary/Spry.
