  The emergence of foundation models, such as the Segment Anything Model (SAM),
has sparked interest in Parameter-Efficient Fine-Tuning (PEFT) methods that
tailor these large models to application domains outside their training data.
However, different PEFT techniques modify the representation of a model
differently, making it a non-trivial task to select the most appropriate method
for the domain of interest. We propose a new framework, Mixture-of-PEFTs
methods (MoPEFT), that is inspired by traditional Mixture-of-Experts (MoE)
methodologies and is utilized for fine-tuning SAM. Our MoPEFT framework
incorporates three different PEFT techniques as submodules and dynamically
learns to activate the ones that are best suited for a given data-task setup.
We test our method on the Segment Anything Model and show that MoPEFT
consistently outperforms other fine-tuning methods on the MESS benchmark.
