  The performance of deep neural network-based speech enhancement systems
typically increases with the training dataset size. However, studies that
investigated the effect of training dataset size on speech enhancement
performance did not consider recent approaches, such as diffusion-based
generative models. Diffusion models are typically trained with massive datasets
for image generation tasks, but whether this is also required for speech
enhancement is unknown. Moreover, studies that investigated the effect of
training dataset size did not control for the data diversity. It is thus
unclear whether the performance improvement was due to the increased dataset
size or diversity. Therefore, we systematically investigate the effect of
training dataset size on the performance of popular state-of-the-art
discriminative and diffusion-based speech enhancement systems. We control for
the data diversity by using a fixed set of speech utterances, noise segments
and binaural room impulse responses to generate datasets of different sizes. We
find that the diffusion-based systems do not benefit from increasing the
training dataset size as much as the discriminative systems. They perform the
best relative to the discriminative systems with datasets of 10 h or less, but
they are outperformed by the discriminative systems with datasets of 100 h or
more.
