  Despite recent advances in Natural Language Processing (NLP), hierarchical
discourse parsing in the framework of Rhetorical Structure Theory remains
challenging, and our understanding of the reasons for this are as yet limited.
In this paper, we examine and model some of the factors associated with parsing
difficulties in previous work: the existence of implicit discourse relations,
challenges in identifying long-distance relations, out-of-vocabulary items, and
more. In order to assess the relative importance of these variables, we also
release two annotated English test-sets with explicit correct and distracting
discourse markers associated with gold standard RST relations. Our results show
that as in shallow discourse parsing, the explicit/implicit distinction plays a
role, but that long-distance dependencies are the main challenge, while lack of
lexical overlap is less of a problem, at least for in-domain parsing. Our final
model is able to predict where errors will occur with an accuracy of 76.3% for
the bottom-up parser and 76.6% for the top-down parser.
