  Training data for text classification is often limited in practice,
especially for applications with many output classes or involving many related
classification problems. This means classifiers must generalize from limited
evidence, but the manner and extent of generalization is task dependent.
Current practice primarily relies on pre-trained word embeddings to map words
unseen in training to similar seen ones. Unfortunately, this squishes many
components of meaning into highly restricted capacity. Our alternative begins
with sparse pre-trained representations derived from unlabeled parsed corpora;
based on the available training data, we select features that offers the
relevant generalizations. This produces task-specific semantic vectors; here,
we show that a feed-forward network over these vectors is especially effective
in low-data scenarios, compared to existing state-of-the-art methods. By
further pairing this network with a convolutional neural network, we keep this
edge in low data scenarios and remain competitive when using full training
sets.
