  We present a feature vector formation technique for documents - Sparse
Composite Document Vector (SCDV) - which overcomes several shortcomings of the
current distributional paragraph vector representations that are widely used
for text representation. In SCDV, word embedding's are clustered to capture
multiple semantic contexts in which words occur. They are then chained together
to form document topic-vectors that can express complex, multi-topic documents.
Through extensive experiments on multi-class and multi-label classification
tasks, we outperform the previous state-of-the-art method, NTSG (Liu et al.,
2015a). We also show that SCDV embedding's perform well on heterogeneous tasks
like Topic Coherence, context-sensitive Learning and Information Retrieval.
Moreover, we achieve significant reduction in training and prediction times
compared to other representation methods. SCDV achieves best of both worlds -
better performance with lower time and space complexity.
