  We present a new topic model that generates documents by sampling a topic for
one whole sentence at a time, and generating the words in the sentence using an
RNN decoder that is conditioned on the topic of the sentence. We argue that
this novel formalism will help us not only visualize and model the topical
discourse structure in a document better, but also potentially lead to more
interpretable topics since we can now illustrate topics by sampling
representative sentences instead of bag of words or phrases. We present a
variational auto-encoder approach for learning in which we use a factorized
variational encoder that independently models the posterior over topical
mixture vectors of documents using a feed-forward network, and the posterior
over topic assignments to sentences using an RNN. Our preliminary experiments
on two different datasets indicate early promise, but also expose many
challenges that remain to be addressed.
