  Existing large language models (LLMs) that mainly focus on Standard American
English (SAE) often lead to significantly worse performance when being applied
to other English dialects. While existing mitigations tackle discrepancies for
individual target dialects, they assume access to high-accuracy dialect
identification systems. The boundaries between dialects are inherently
flexible, making it difficult to categorize language into discrete predefined
categories. In this paper, we propose DADA (Dialect Adaptation via Dynamic
Aggregation), a modular approach to imbue SAE-trained models with
multi-dialectal robustness by composing adapters which handle specific
linguistic features. The compositional architecture of DADA allows for both
targeted adaptation to specific dialect variants and simultaneous adaptation to
various dialects. We show that DADA is effective for both single task and
instruction finetuned language models, offering an extensible and interpretable
framework for adapting existing LLMs to different English dialects.
