  In this paper, we propose MM-KWS, a novel approach to user-defined keyword
spotting leveraging multi-modal enrollments of text and speech templates.
Unlike previous methods that focus solely on either text or speech features,
MM-KWS extracts phoneme, text, and speech embeddings from both modalities.
These embeddings are then compared with the query speech embedding to detect
the target keywords. To ensure the applicability of MM-KWS across diverse
languages, we utilize a feature extractor incorporating several multilingual
pre-trained models. Subsequently, we validate its effectiveness on Mandarin and
English tasks. In addition, we have integrated advanced data augmentation tools
for hard case mining to enhance MM-KWS in distinguishing confusable words.
Experimental results on the LibriPhrase and WenetPhrase datasets demonstrate
that MM-KWS outperforms prior methods significantly.
