  Low-Rank Adaptation (LoRA) has gained popularity for fine-tuning large
foundation models, leveraging low-rank matrices $\mathbf{A}$ and $\mathbf{B}$
to represent weight changes (\textit{i.e.,} $\Delta \mathbf{W} = \mathbf{B}
\mathbf{A}$). This method reduces trainable parameters and mitigates heavy
memory consumption associated with full delta matrices by sequentially
multiplying $\mathbf{A}$ and $\mathbf{B}$ with the activation. Despite its
success, the intrinsic low-rank characteristic may limit its performance.
Although several variants have been proposed to address this issue, they often
overlook the crucial computational and memory efficiency brought by LoRA. In
this paper, we propose \underline{C}ir\underline{c}ular \underline{C}onvolution
\underline{A}daptation (C$^3$A), which not only achieves high-rank adaptation
with enhanced performance but also excels in both computational power and
memory utilization. Extensive experiments demonstrate that C$^3$A consistently
outperforms LoRA and its variants across various fine-tuning tasks.
