  While much work on deep latent variable models of text uses continuous latent
variables, discrete latent variables are interesting because they are more
interpretable and typically more space efficient. We consider several
approaches to learning discrete latent variable models for text in the case
where exact marginalization over these variables is intractable. We compare the
performance of the learned representations as features for low-resource
document and sentence classification. Our best models outperform the previous
best reported results with continuous representations in these low-resource
settings, while learning significantly more compressed representations.
Interestingly, we find that an amortized variant of Hard EM performs
particularly well in the lowest-resource regimes.
