  The prevailing practice in the academia is to evaluate the model performance
on in-domain evaluation data typically set aside from the training corpus.
However, in many real world applications the data on which the model is applied
may very substantially differ from the characteristics of the training data. In
this paper, we focus on Finnish out-of-domain parsing by introducing a novel UD
Finnish-OOD out-of-domain treebank including five very distinct data sources
(web documents, clinical, online discussions, tweets, and poetry), and a total
of 19,382 syntactic words in 2,122 sentences released under the Universal
Dependencies framework. Together with the new treebank, we present extensive
out-of-domain parsing evaluation utilizing the available section-level
information from three different Finnish UD treebanks (TDT, PUD, OOD). Compared
to the previously existing treebanks, the new Finnish-OOD is shown include
sections more challenging for the general parser, creating an interesting
evaluation setting and yielding valuable information for those applying the
parser outside of its training domain.
