  The vast majority of textual content is unstructured, making automated
classification an important task for many applications. The goal of text
classification is to automatically classify text documents into one or more
predefined categories. Recently proposed simple architectures for text
classification such as Convolutional Neural Networks for Sentence
Classification by Kim, Yoon showed promising results. In this paper, we propose
incorporating an attention mechanism into the network to boost its performance,
we also propose WordRank for vocabulary selection to reduce the network
embedding parameters and speed up training with minimum accuracy loss. By
adopting the proposed ideas TextCNN accuracy on 20News increased from 94.79 to
96.88, moreover, the number of parameters for the embedding layer can be
reduced substantially with little accuracy loss by using WordRank. By using
WordRank for vocabulary selection we can reduce the number of parameters by
more than 5x from 7.9M to 1.5M, and the accuracy will only decrease by 1.2%.
