  Nowadays, with the booming development of the Internet, people benefit from
its convenience due to its open and sharing nature. A large volume of natural
language texts is being generated by users in various forms, such as search
queries, documents, and social media posts. As the unstructured text corpus is
usually noisy and messy, it becomes imperative to correctly identify and
accurately annotate structured information in order to obtain meaningful
insights or better understand unstructured texts. On the other hand, the
existing structured information, which embodies our knowledge such as entity or
concept relations, often suffers from incompleteness or quality-related issues.
Given a gigantic collection of texts which offers rich semantic information, it
is also important to harness the massiveness of the unannotated text corpus to
expand and refine existing structured knowledge with fewer annotation efforts.
  In this dissertation, I will introduce principles, models, and algorithms for
effective structured knowledge discovery from the massive text corpus. We are
generally interested in obtaining insights and better understanding
unstructured texts with the help of structured annotations or by
structure-aware modeling. Also, given the existing structured knowledge, we are
interested in expanding its scale and improving its quality harnessing the
massiveness of the text corpus. In particular, four problems are studied in
this dissertation: Structured Intent Detection for Natural Language
Understanding, Structure-aware Natural Language Modeling, Generative Structured
Knowledge Expansion, and Synonym Refinement on Structured Knowledge.
