  Foundation models (FoMos), referring to large-scale AI models, possess
human-like capabilities and are able to perform competitively in the domain of
human intelligence. The breakthrough in FoMos has inspired researchers to
deploy such models in the sixth-generation (6G) mobile networks for automating
a broad range of tasks in next-generation mobile applications. While the sizes
of FoMos are reaching their peaks, their next phase is expected to focus on
fine-tuning the models to specific downstream tasks. This inspires us to
propose the vision of FoMo fine-tuning as a 6G service. Its key feature is the
exploitation of existing parameter-efficient fine-tuning (PEFT) techniques to
tweak only a small fraction of model weights for a FoMo to become customized
for a specific task. To materialize the said vision, we survey the
state-of-the-art PEFT and then present a novel device-edge fine-tuning (DEFT)
framework for providing efficient and privacy-preserving fine-tuning services
at the 6G network edge. The framework consists of the following comprehensive
set of techniques: 1) Control of fine-tuning parameter sizes in different
transformer blocks of a FoMo; 2) Over-the-air computation for realizing neural
connections in DEFT; 3) Federated DEFT in a multi-device system by downloading
a FoMo emulator or gradients; 4) On-the-fly prompt-ensemble tuning; 5)
Device-to-device prompt transfer among devices. Experiments are conducted using
pre-trained FoMos with up to 11 billion parameters to demonstrate the
effectiveness of DEFT techniques. The article is concluded by presenting future
research opportunities.
