  Speech enhancement improves speech quality and promotes the performance of
various downstream tasks. However, most current speech enhancement work was
mainly devoted to improving the performance of downstream automatic speech
recognition (ASR), only a relatively small amount of work focused on the
automatic speaker verification (ASV) task. In this work, we propose a MVNet
consisted of a memory assistance module which improves the performance of
downstream ASR and a vocal reinforcement module which boosts the performance of
ASV. In addition, we design a new loss function to improve speaker vocal
similarity. Experimental results on the Libri2mix dataset show that our method
outperforms baseline methods in several metrics, including speech quality,
intelligibility, and speaker vocal similarity et al.
