  Unsupervised text embedding has shown great power in a wide range of NLP
tasks. While text embeddings are typically learned in the Euclidean space,
directional similarity is often more effective in tasks such as word similarity
and document clustering, which creates a gap between the training stage and
usage stage of text embedding. To close this gap, we propose a spherical
generative model based on which unsupervised word and paragraph embeddings are
jointly learned. To learn text embeddings in the spherical space, we develop an
efficient optimization algorithm with convergence guarantee based on Riemannian
optimization. Our model enjoys high efficiency and achieves state-of-the-art
performances on various text embedding tasks including word similarity and
document clustering.
