  Multi-Source Domain Adaptation (MSDA) is a challenging scenario where
multiple related and heterogeneous source datasets must be adapted to an
unlabeled target dataset. Conventional MSDA methods often overlook that data
holders may have privacy concerns, hindering direct data sharing. In response,
decentralized MSDA has emerged as a promising strategy to achieve adaptation
without centralizing clients' data. Our work proposes a novel approach,
Decentralized Dataset Dictionary Learning, to address this challenge. Our
method leverages Wasserstein barycenters to model the distributional shift
across multiple clients, enabling effective adaptation while preserving data
privacy. Specifically, our algorithm expresses each client's underlying
distribution as a Wasserstein barycenter of public atoms, weighted by private
barycentric coordinates. Our approach ensures that the barycentric coordinates
remain undisclosed throughout the adaptation process. Extensive experimentation
across five visual domain adaptation benchmarks demonstrates the superiority of
our strategy over existing decentralized MSDA techniques. Moreover, our method
exhibits enhanced robustness to client parallelism while maintaining relative
resilience compared to conventional decentralized MSDA methodologies.
