  We propose a method to fuse posterior distributions learned from
heterogeneous datasets. Our algorithm relies on a mean field assumption for
both the fused model and the individual dataset posteriors and proceeds using a
simple assign-and-average approach. The components of the dataset posteriors
are assigned to the proposed global model components by solving a regularized
variant of the assignment problem. The global components are then updated based
on these assignments by their mean under a KL divergence. For exponential
family variational distributions, our formulation leads to an efficient
non-parametric algorithm for computing the fused model. Our algorithm is easy
to describe and implement, efficient, and competitive with state-of-the-art on
motion capture analysis, topic modeling, and federated learning of Bayesian
neural networks.
