  VQ-VAE, as a mainstream approach of speech tokenizer, has been troubled by
``index collapse'', where only a small number of codewords are activated in
large codebooks. This work proposes product-quantized (PQ) VAE with more
codebooks but fewer codewords to address this problem and build large-codebook
speech tokenizers. It encodes speech features into multiple VQ subspaces and
composes them into codewords in a larger codebook. Besides, to utilize each VQ
subspace well, we also enhance PQ-VAE via a dual-decoding training strategy
with the encoding and quantized sequences. The experimental results demonstrate
that PQ-VAE addresses ``index collapse" effectively, especially for larger
codebooks. The model with the proposed training strategy further improves
codebook perplexity and reconstruction quality, outperforming other
multi-codebook VQ approaches. Finally, PQ-VAE demonstrates its effectiveness in
language-model-based TTS, supporting higher-quality speech generation with
larger codebooks.
