  Convolutional neural network (CNN) and recurrent neural network (RNN) are two
popular architectures used in text classification. Traditional methods to
combine the strengths of the two networks rely on streamlining them or
concatenating features extracted from them. In this paper, we propose a novel
method to keep the strengths of the two networks to a great extent. In the
proposed model, a convolutional neural network is applied to learn a 2D weight
matrix where each row reflects the importance of each word from different
aspects. Meanwhile, we use a bi-directional RNN to process each word and employ
a neural tensor layer that fuses forward and backward hidden states to get word
representations. In the end, the weight matrix and word representations are
combined to obtain the representation in a 2D matrix form for the text. We
carry out experiments on a number of datasets for text classification. The
experimental results confirm the effectiveness of the proposed method.
