  In this paper, we investigate representation learning for low-resource
keyword spotting (KWS). The main challenges of KWS are limited labeled data and
limited available device resources. To address those challenges, we explore
representation learning for KWS by self-supervised contrastive learning and
self-training with pretrained model. First, local-global contrastive siamese
networks (LGCSiam) are designed to learn similar utterance-level
representations for similar audio samplers by proposed local-global contrastive
loss without requiring ground-truth. Second, a self-supervised pretrained
Wav2Vec 2.0 model is applied as a constraint module (WVC) to force the KWS
model to learn frame-level acoustic representations. By the LGCSiam and WVC
modules, the proposed small-footprint KWS model can be pretrained with
unlabeled data. Experiments on speech commands dataset show that the
self-training WVC module and the self-supervised LGCSiam module significantly
improve accuracy, especially in the case of training on a small labeled
dataset.
