  Embedding from Language Models (ELMo) has shown to be effective for improving
many natural language processing (NLP) tasks, and ELMo takes character
information to compose word representation to train language models.However,
the character is an insufficient and unnatural linguistic unit for word
representation.Thus we introduce Embedding from Subword-aware Language Models
(ESuLMo) which learns word representation from subwords using unsupervised
segmentation over words.We show that ESuLMo can enhance four benchmark NLP
tasks more effectively than ELMo, including syntactic dependency parsing,
semantic role labeling, implicit discourse relation recognition and textual
entailment, which brings a meaningful improvement over ELMo.
