  This article deals with the problem of distributed machine learning, in which
agents update their models based on their local datasets, and aggregate the
updated models collaboratively and in a fully decentralized manner. In this
paper, we tackle the problem of information heterogeneity arising in
multi-agent networks where the placement of informative agents plays a crucial
role in the learning dynamics. Specifically, we propose BayGo, a novel fully
decentralized joint Bayesian learning and graph optimization framework with
proven fast convergence over a sparse graph. Under our framework, agents are
able to learn and communicate with the most informative agent to their own
learning. Unlike prior works, our framework assumes no prior knowledge of the
data distribution across agents nor does it assume any knowledge of the true
parameter of the system. The proposed alternating minimization based framework
ensures global connectivity in a fully decentralized way while minimizing the
number of communication links. We theoretically show that by optimizing the
proposed objective function, the estimation error of the posterior probability
distribution decreases exponentially at each iteration. Via extensive
simulations, we show that our framework achieves faster convergence and higher
accuracy compared to fully-connected and star topology graphs.
