  Recent advancements in large language models (LLMs) have indeed showcased
their impressive capabilities. On mobile devices, the wealth of valuable,
non-public data generated daily holds great promise for locally fine-tuning
personalized LLMs, while maintaining privacy through on-device processing.
However, the constraints of mobile device resources pose challenges to direct
on-device LLM fine-tuning, mainly due to the memory-intensive nature of
derivative-based optimization required for saving gradients and optimizer
states. To tackle this, we propose employing derivative-free optimization
techniques to enable on-device fine-tuning of LLM, even on memory-limited
mobile devices. Empirical results demonstrate that the RoBERTa-large model and
OPT-1.3B can be fine-tuned locally on the OPPO Reno 6 smartphone using around
4GB and 6.5GB of memory respectively, using derivative-free optimization
techniques. This highlights the feasibility of on-device LLM fine-tuning on
mobile devices, paving the way for personalized LLMs on resource-constrained
devices while safeguarding data privacy.
