  The speaker extraction technique seeks to single out the voice of a target
speaker from the interfering voices in a speech mixture. Typically an auxiliary
reference of the target speaker is used to form voluntary attention. Either a
pre-recorded utterance or a synchronized lip movement in a video clip can serve
as the auxiliary reference. The use of visual cue is not only feasible, but
also effective due to its noise robustness, and becoming popular. However, it
is difficult to guarantee that such parallel visual cue is always available in
real-world applications where visual occlusion or intermittent communication
can occur. In this paper, we study the audio-visual speaker extraction
algorithms with intermittent visual cue. We propose a joint speaker extraction
and visual embedding inpainting framework to explore the mutual benefits. To
encourage the interaction between the two tasks, they are performed alternately
with an interlacing structure and optimized jointly. We also propose two types
of visual inpainting losses and study our proposed method with two types of
popularly used visual embeddings. The experimental results show that we
outperform the baseline in terms of signal quality, perceptual quality, and
intelligibility.
