  Embedding layers are commonly used to map discrete symbols into continuous
embedding vectors that reflect their semantic meanings. Despite their
effectiveness, the number of parameters in an embedding layer increases
linearly with the number of symbols and poses a critical challenge on memory
and storage constraints. In this work, we propose a generic and end-to-end
learnable compression framework termed differentiable product quantization
(DPQ). We present two instantiations of DPQ that leverage different
approximation techniques to enable differentiability in end-to-end learning.
Our method can readily serve as a drop-in alternative for any existing
embedding layer. Empirically, DPQ offers significant compression ratios
(14-238$\times$) at negligible or no performance cost on 10 datasets across
three different language tasks.
