  We present LatinPipe, the winning submission to the EvaLatin 2024 Dependency
Parsing shared task. Our system consists of a fine-tuned concatenation of base
and large pre-trained LMs, with a dot-product attention head for parsing and
softmax classification heads for morphology to jointly learn both dependency
parsing and morphological analysis. It is trained by sampling from seven
publicly available Latin corpora, utilizing additional harmonization of
annotations to achieve a more unified annotation style. Before fine-tuning, we
train the system for a few initial epochs with frozen weights. We also add
additional local relative contextualization by stacking the BiLSTM layers on
top of the Transformer(s). Finally, we ensemble output probability
distributions from seven randomly instantiated networks for the final
submission. The code is available at
https://github.com/ufal/evalatin2024-latinpipe.
