  The classification of sentences is very challenging, since sentences contain
the limited contextual information. In this paper, we proposed an
Attention-Gated Convolutional Neural Network (AGCNN) for sentence
classification, which generates attention weights from the feature's context
windows of different sizes by using specialized convolution encoders. It makes
full use of limited contextual information to extract and enhance the influence
of important features in predicting the sentence's category. Experimental
results demonstrated that our model can achieve up to 3.1% higher accuracy than
standard CNN models, and gain competitive results over the baselines on four
out of the six tasks. Besides, we designed an activation function, namely,
Natural Logarithm rescaled Rectified Linear Unit (NLReLU). Experiments showed
that NLReLU can outperform ReLU and is comparable to other well-known
activation functions on AGCNN.
