  Morphological information is important for many sequence labeling tasks in
Natural Language Processing (NLP). Yet, existing approaches rely heavily on
manual annotations or external software to capture this information. In this
study, we propose using subword contextual embeddings to capture the
morphological information for languages with rich morphology. In addition, we
incorporate these embeddings in a hierarchical multi-task setting which is not
employed before, to the best of our knowledge. Evaluated on Dependency Parsing
(DEP) and Named Entity Recognition (NER) tasks, which are shown to benefit
greatly from morphological information, our final model outperforms previous
state-of-the-art models on both tasks for the Turkish language. Besides, we
show a net improvement of 18.86% and 4.61% F-1 over the previously proposed
multi-task learner in the same setting for the DEP and the NER tasks,
respectively. Empirical results for five different MTL settings show that
incorporating subword contextual embeddings brings significant improvements for
both tasks. In addition, we observed that multi-task learning consistently
improves the performance of the DEP component.
