  Documents are composed of smaller pieces - paragraphs, sentences, and tokens
- that have complex relationships between one another. Sentiment classification
models that take into account the structure inherent in these documents have a
theoretical advantage over those that do not. At the same time, transfer
learning models based on language model pretraining have shown promise for
document classification. However, these two paradigms have not been
systematically compared and it is not clear under which circumstances one
approach is better than the other. In this work we empirically compare
hierarchical models and transfer learning for document-level sentiment
classification. We show that non-trivial hierarchical models outperform
previous baselines and transfer learning on document-level sentiment
classification in five languages.
