  Recently, researchers have shown an increasing interest in automatically
predicting the subjective evaluation for speech synthesis systems. This
prediction is a challenging task, especially on the out-of-domain test set. In
this paper, we proposed a novel fusion model for MOS prediction that combines
supervised and unsupervised approaches. In the supervised aspect, we developed
an SSL-based predictor called LE-SSL-MOS. The LE-SSL-MOS utilizes pre-trained
self-supervised learning models and further improves prediction accuracy by
utilizing the opinion scores of each utterance in the listener enhancement
branch. In the unsupervised aspect, two steps are contained: we fine-tuned the
unit language model (ULM) using highly intelligible domain data to improve the
correlation of an unsupervised metric - SpeechLMScore. Another is that we
utilized ASR confidence as a new metric with the help of ensemble learning. To
our knowledge, this is the first architecture that fuses supervised and
unsupervised methods for MOS prediction. With these approaches, our
experimental results on the VoiceMOS Challenge 2023 show that LE-SSL-MOS
performs better than the baseline. Our fusion system achieved an absolute
improvement of 13% over LE-SSL-MOS on the noisy and enhanced speech track. Our
system ranked 1st and 2nd, respectively, in the French speech synthesis track
and the challenge's noisy and enhanced speech track.
