  Transfer learning (TL) is an increasingly popular approach to training deep
learning (DL) models that leverages the knowledge gained by training a
foundation model on diverse, large-scale datasets for use on downstream tasks
where less domain- or task-specific data is available. The literature is rich
with TL techniques and applications; however, the bulk of the research makes
use of deterministic DL models which are often uncalibrated and lack the
ability to communicate a measure of epistemic (model) uncertainty in
prediction. Unlike their deterministic counterparts, Bayesian DL (BDL) models
are often well-calibrated, provide access to epistemic uncertainty for a
prediction, and are capable of achieving competitive predictive performance. In
this study, we propose variational inference pre-trained audio neural networks
(VI-PANNs). VI-PANNs are a variational inference variant of the popular
ResNet-54 architecture which are pre-trained on AudioSet, a large-scale audio
event detection dataset. We evaluate the quality of the resulting uncertainty
when transferring knowledge from VI-PANNs to other downstream acoustic
classification tasks using the ESC-50, UrbanSound8K, and DCASE2013 datasets. We
demonstrate, for the first time, that it is possible to transfer calibrated
uncertainty information along with knowledge from upstream tasks to enhance a
model's capability to perform downstream tasks.
