  We propose a new unsupervised model for mapping a variable-duration speech
segment to a fixed-dimensional representation. The resulting acoustic word
embeddings can form the basis of search, discovery, and indexing systems for
low- and zero-resource languages. Our model, which we refer to as a maximal
sampling correspondence variational autoencoder (MCVAE), is a recurrent neural
network (RNN) trained with a novel self-supervised correspondence loss that
encourages consistency between embeddings of different instances of the same
word. Our training scheme improves on previous correspondence training
approaches through the use and comparison of multiple samples from the
approximate posterior distribution. In the zero-resource setting, the MCVAE can
be trained in an unsupervised way, without any ground-truth word pairs, by
using the word-like segments discovered via an unsupervised term discovery
system. In both this setting and a semi-supervised low-resource setting (with a
limited set of ground-truth word pairs), the MCVAE outperforms previous
state-of-the-art models, such as Siamese-, CAE- and VAE-based RNNs.
