  Estimating noise information exactly is crucial for noise aware training in
speech applications including speech enhancement (SE) which is our focus in
this paper. To estimate noise-only frames, we employ voice activity detection
(VAD) to detect non-speech frames by applying optimal threshold on speech
posterior. Here, the non-speech frames can be regarded as noise-only frames in
noisy signal. These estimated frames are used to extract noise embedding, named
dynamic noise embedding (DNE), which is useful for an SE module to capture the
characteristic of background noise. The DNE is extracted by a simple neural
network, and the SE module with the DNE can be jointly trained to be adaptive
to the environment. Experiments are conducted on TIMIT dataset for
single-channel denoising task and U-Net is used as a backbone SE module.
Experimental results show that the DNE plays an important role in the SE module
by increasing the quality and the intelligibility of corrupted signal even if
the noise is non-stationary and unseen in training. In addition, we demonstrate
that the DNE can be flexibly applied to other neural network-based SE modules.
