  The bag-of-words model is a standard representation of text for many linear
classifier learners. In many problem domains, linear classifiers are preferred
over more complex models due to their efficiency, robustness and
interpretability, and the bag-of-words text representation can capture
sufficient information for linear classifiers to make highly accurate
predictions. However in settings where there is a large vocabulary, large
variance in the frequency of terms in the training corpus, many classes and
very short text (e.g., single sentences or document titles) the bag-of-words
representation becomes extremely sparse, and this can reduce the accuracy of
classifiers. A particular issue in such settings is that short texts tend to
contain infrequently occurring or rare terms which lack class-conditional
evidence. In this work we introduce a method for enriching the bag-of-words
model by complementing such rare term information with related terms from both
general and domain-specific Word Vector models. By reducing sparseness in the
bag-of-words models, our enrichment approach achieves improved classification
over several baseline classifiers in a variety of text classification problems.
Our approach is also efficient because it requires no change to the linear
classifier before or during training, since bag-of-words enrichment applies
only to text being classified.
