  Although semi-supervised variational autoencoder (SemiVAE) works in image
classification task, it fails in text classification task if using vanilla LSTM
as its decoder. From a perspective of reinforcement learning, it is verified
that the decoder's capability to distinguish between different categorical
labels is essential. Therefore, Semi-supervised Sequential Variational
Autoencoder (SSVAE) is proposed, which increases the capability by feeding
label into its decoder RNN at each time-step. Two specific decoder structures
are investigated and both of them are verified to be effective. Besides, in
order to reduce the computational complexity in training, a novel optimization
method is proposed, which estimates the gradient of the unlabeled objective
function by sampling, along with two variance reduction techniques.
Experimental results on Large Movie Review Dataset (IMDB) and AG's News corpus
show that the proposed approach significantly improves the classification
accuracy compared with pure-supervised classifiers, and achieves competitive
performance against previous advanced methods. State-of-the-art results can be
obtained by integrating other pretraining-based methods.
