  Speech enhancement techniques improve the quality or the intelligibility of
an audio signal by removing unwanted noise. It is used as preprocessing in
numerous applications such as speech recognition, hearing aids, broadcasting
and telephony. The evaluation of such algorithms often relies on
reference-based objective metrics that are shown to correlate poorly with human
perception. In order to evaluate audio quality as perceived by human observers
it is thus fundamental to resort to subjective quality assessment. In this
paper, a user evaluation based on crowdsourcing (subjective) and the Comparison
Category Rating (CCR) method is compared against the DNSMOS, ViSQOL and 3QUEST
(objective) metrics. The overall quality scores of three speech enhancement
algorithms from real time communications (RTC) are used in the comparison using
the P.808 toolkit. Results indicate that while the CCR scale allows
participants to identify differences between processed and unprocessed audio
samples, two groups of preferences emerge: some users rate positively by
focusing on noise suppression processing, while others rate negatively by
focusing mainly on speech quality. We further present results on the
parameters, size considerations and speaker variations that are critical and
should be considered when designing the CCR-based crowdsourcing evaluation.
