  Speaker-aware source separation methods are promising workarounds for major
difficulties such as arbitrary source permutation and unknown number of
sources. However, it remains challenging to achieve satisfying performance
provided a very short available target speaker utterance (anchor). Here we
present a novel "deep extractor network" which creates an extractor point for
the target speaker in a canonical high dimensional embedding space, and pulls
together the time-frequency bins corresponding to the target speaker. The
proposed model is different from prior works in that the canonical embedding
space encodes knowledges of both the anchor and the mixture during an
end-to-end training phase: First, embeddings for the anchor and mixture speech
are separately constructed in a primary embedding space, and then combined as
an input to feed-forward layers to transform to a canonical embedding space
which we discover more stable than the primary one. Experimental results show
that given a very short utterance, the proposed model can efficiently recover
high quality target speech from a mixture, which outperforms various baseline
models, with 5.2% and 6.6% relative improvements in SDR and PESQ respectively
compared with a baseline oracle deep attracor model. Meanwhile, we show it can
be generalized well to more than one interfering speaker.
