  In this paper we describe our entry for the VoiceMOS Challenge 2022 for both
the main and out-of-domain (OOD) track of the competition. Our system is based
on finetuning pre-trained self-supervised waveform prediction models, while
improving its generalisation ability through stochastic weight averaging.
Further, we use influence functions to identity possible low-quality data
within the training set to further increase our model's performance for the OOD
track. Our system ranked 5th and joint 7th for the main track and OOD track,
respectively.
