  In modern federated learning, one of the main challenges is to account for
inherent heterogeneity and the diverse nature of data distributions for
different clients. This problem is often addressed by introducing
personalization of the models towards the data distribution of the particular
client. However, a personalized model might be unreliable when applied to the
data that is not typical for this client. Eventually, it may perform worse for
these data than the non-personalized global model trained in a federated way on
the data from all the clients. This paper presents a new approach to federated
learning that allows selecting a model from global and personalized ones that
would perform better for a particular input point. It is achieved through a
careful modeling of predictive uncertainties that helps to detect local and
global in- and out-of-distribution data and use this information to select the
model that is confident in a prediction. The comprehensive experimental
evaluation on the popular real-world image datasets shows the superior
performance of the model in the presence of out-of-distribution data while
performing on par with state-of-the-art personalized federated learning
algorithms in the standard scenarios.
