  Transfer learning has recently shown significant performance across various
tasks involving deep neural networks. In these transfer learning scenarios, the
prior distribution for downstream data becomes crucial in Bayesian model
averaging (BMA). While previous works proposed the prior over the neural
network parameters centered around the pre-trained solution, such strategies
have limitations when dealing with distribution shifts between upstream and
downstream data. This paper introduces nonparametric transfer learning (NPTL),
a flexible posterior sampling method to address the distribution shift issue
within the context of nonparametric learning. The nonparametric learning (NPL)
method is a recent approach that employs a nonparametric prior for posterior
sampling, efficiently accounting for model misspecification scenarios, which is
suitable for transfer learning scenarios that may involve the distribution
shift between upstream and downstream tasks. Through extensive empirical
validations, we demonstrate that our approach surpasses other baselines in BMA
performance.
