  Past work that improves document-level sentiment analysis by encoding user
and product information has been limited to considering only the text of the
current review. We investigate incorporating additional review text available
at the time of sentiment prediction that may prove meaningful for guiding
prediction. Firstly, we incorporate all available historical review text
belonging to the author of the review in question. Secondly, we investigate the
inclusion of historical reviews associated with the current product (written by
other users). We achieve this by explicitly storing representations of reviews
written by the same user and about the same product and force the model to
memorize all reviews for one particular user and product. Additionally, we drop
the hierarchical architecture used in previous work to enable words in the text
to directly attend to each other. Experiment results on IMDB, Yelp 2013 and
Yelp 2014 datasets show improvement to state-of-the-art of more than 2
percentage points in the best case.
