  Much work in Natural Language Processing (NLP) has been for resource-rich
languages, making generalization to new, less-resourced languages challenging.
We present two approaches for improving generalization to low-resourced
languages by adapting continuous word representations using linguistically
motivated subword units: phonemes, morphemes and graphemes. Our method requires
neither parallel corpora nor bilingual dictionaries and provides a significant
gain in performance over previous methods relying on these resources. We
demonstrate the effectiveness of our approaches on Named Entity Recognition for
four languages, namely Uyghur, Turkish, Bengali and Hindi, of which Uyghur and
Bengali are low resource languages, and also perform experiments on Machine
Translation. Exploiting subwords with transfer learning gives us a boost of
+15.2 NER F1 for Uyghur and +9.7 F1 for Bengali. We also show improvements in
the monolingual setting where we achieve (avg.) +3 F1 and (avg.) +1.35 BLEU.
