  We propose a method to learn unsupervised sentence representations in a
non-compositional manner based on Generative Latent Optimization. Our approach
does not impose any assumptions on how words are to be combined into a sentence
representation. We discuss a simple Bag of Words model as well as a variant
that models word positions. Both are trained to reconstruct the sentence based
on a latent code and our model can be used to generate text. Experiments show
large improvements over the related Paragraph Vectors. Compared to uSIF, we
achieve a relative improvement of 5% when trained on the same data and our
method performs competitively to Sent2vec while trained on 30 times less data.
