  With the advances in deep learning, speech enhancement systems benefited from
large neural network architectures and achieved state-of-the-art quality.
However, speaker-agnostic methods are not always desirable, both in terms of
quality and their complexity, when they are to be used in a
resource-constrained environment. One promising way is personalized speech
enhancement (PSE), which is a smaller and easier speech enhancement problem for
small models to solve, because it focuses on a particular test-time user. To
achieve the personalization goal, while dealing with the typical lack of
personal data, we investigate the effect of data augmentation based on neural
speech synthesis (NSS). In the proposed method, we show that the quality of the
NSS system's synthetic data matters, and if they are good enough the augmented
dataset can be used to improve the PSE system that outperforms the
speaker-agnostic baseline. The proposed PSE systems show significant complexity
reduction while preserving the enhancement quality.
