  Recently, neural networks have achieved great success on sentiment
classification due to their ability to alleviate feature engineering. However,
one of the remaining challenges is to model long texts in document-level
sentiment classification under a recurrent architecture because of the
deficiency of the memory unit. To address this problem, we present a Cached
Long Short-Term Memory neural networks (CLSTM) to capture the overall semantic
information in long texts. CLSTM introduces a cache mechanism, which divides
memory into several groups with different forgetting rates and thus enables the
network to keep sentiment information better within a recurrent unit. The
proposed CLSTM outperforms the state-of-the-art models on three publicly
available document-level sentiment analysis datasets.
