  Efficient adaption of large language models (LLMs) on edge devices is
essential for applications requiring continuous and privacy-preserving
adaptation and inference. However, existing tuning techniques fall short
because of the high computation and memory overheads. To this end, we introduce
a computation- and memory-efficient LLM tuning framework, called Edge-LLM, to
facilitate affordable and effective LLM adaptation on edge devices.
Specifically, Edge-LLM features three core components: (1) a layer-wise unified
compression (LUC) technique to reduce the computation overhead by generating
layer-wise pruning sparsity and quantization bit-width policies, (2) an
adaptive layer tuning and voting scheme to reduce the memory overhead by
reducing the backpropagation depth, and (3) a complementary hardware scheduling
strategy to handle the irregular computation patterns introduced by LUC and
adaptive layer tuning, thereby achieving efficient computation and data
movements. Extensive experiments demonstrate that Edge-LLM achieves a 2.92x
speed up and a 4x memory overhead reduction as compared to vanilla tuning
methods with comparable task accuracy. Our code is available at
https://github.com/GATECH-EIC/Edge-LLM
