  In this paper, we study multi-target domain adaptation of scene understanding
models. While previous methods achieved commendable results through
inter-domain consistency losses, they often assumed unrealistic simultaneous
access to images from all target domains, overlooking constraints such as data
transfer bandwidth limitations and data privacy concerns. Given these
challenges, we pose the question: How to merge models adapted independently on
distinct domains while bypassing the need for direct access to training data?
Our solution to this problem involves two components, merging model parameters
and merging model buffers (i.e., normalization layer statistics). For merging
model parameters, empirical analyses of mode connectivity surprisingly reveal
that linear merging suffices when employing the same pretrained backbone
weights for adapting separate models. For merging model buffers, we model the
real-world distribution with a Gaussian prior and estimate new statistics from
the buffers of separately trained models. Our method is simple yet effective,
achieving comparable performance with data combination training baselines,
while eliminating the need for accessing training data. Project page:
https://air-discover.github.io/ModelMerging
