  Speech quality in online conferencing applications is typically assessed
through human judgements in the form of the mean opinion score (MOS) metric.
Since such a labor-intensive approach is not feasible for large-scale speech
quality assessments in most settings, the focus has shifted towards automated
MOS prediction through end-to-end training of deep neural networks (DNN).
Instead of training a network from scratch, we propose to leverage the speech
representations from the pre-trained wav2vec-based XLS-R model. However, the
number of parameters of such a model exceeds task-specific DNNs by several
orders of magnitude, which poses a challenge for resulting fine-tuning
procedures on smaller datasets. Therefore, we opt to use pre-trained speech
representations from XLS-R in a feature extraction rather than a fine-tuning
setting, thereby significantly reducing the number of trainable model
parameters. We compare our proposed XLS-R-based feature extractor to a
Mel-frequency cepstral coefficient (MFCC)-based one, and experiment with
various combinations of bidirectional long short term memory (Bi-LSTM) and
attention pooling feedforward (AttPoolFF) networks trained on the output of the
feature extractors. We demonstrate the increased performance of pre-trained
XLS-R embeddings in terms a reduced root mean squared error (RMSE) on the
ConferencingSpeech 2022 MOS prediction task.
