  Automated Program Repair (APR) has evolved significantly with the advent of
Large Language Models (LLMs). Fine-tuning LLMs for program repair is a recent
avenue of research, with many dimensions which have not been explored. Existing
work mostly fine-tune LLMs with naive code representations and does not scale
to frontier models. To address this problem, we propose RepairLLaMA, a novel
program repair approach that 1) identifies optimal code representations for APR
with fine-tuned models, and 2) pioneers state-of-the-art parameter-efficient
fine-tuning technique (PEFT) for program repair. This results in RepairLLaMA
producing a highly effective `program repair adapter' for fixing bugs with AI.
Our experiments demonstrate the validity of both concepts. First, fine-tuning
adapters with program repair specific code representations enables the model to
use meaningful repair signals and produce better patches. Second,
parameter-efficient fine-tuning helps fine-tuning to converge and clearly
contributes to the effectiveness of RepairLLaMA in fixing bugs outside the
fine-tuning data distribution. Overall, RepairLLaMA correctly fixes 144
Defects4J v2 and 109 HumanEval-Java bugs, outperforming all baselines.
