  Recently, discrete latent variable models have received a surge of interest
in both Natural Language Processing (NLP) and Computer Vision (CV), attributed
to their comparable performance to the continuous counterparts in
representation learning, while being more interpretable in their predictions.
In this paper, we develop a topic-informed discrete latent variable model for
semantic textual similarity, which learns a shared latent space for
sentence-pair representation via vector quantization. Compared with previous
models limited to local semantic contexts, our model can explore richer
semantic information via topic modeling. We further boost the performance of
semantic similarity by injecting the quantized representation into a
transformer-based language model with a well-designed semantic-driven attention
mechanism. We demonstrate, through extensive experiments across various English
language datasets, that our model is able to surpass several strong neural
baselines in semantic textual similarity tasks.
