  As the scale of generative models continues to grow, efficient reuse and
adaptation of pre-trained models have become crucial considerations. In this
work, we propose Voicebox Adapter, a novel approach that integrates
fine-grained conditions into a pre-trained Voicebox speech generation model
using a cross-attention module. To ensure a smooth integration of newly added
modules with pre-trained ones, we explore various efficient fine-tuning
approaches. Our experiment shows that the LoRA with bias-tuning configuration
yields the best performance, enhancing controllability without compromising
speech quality. Across three fine-grained conditional generation tasks, we
demonstrate the effectiveness and resource efficiency of Voicebox Adapter.
Follow-up experiments further highlight the robustness of Voicebox Adapter
across diverse data setups.
