  Language models (LMs) are sentence-completion engines trained on massive
corpora. LMs have emerged as a significant breakthrough in natural-language
processing, providing capabilities that go far beyond sentence completion
including question answering, summarization, and natural-language inference.
While many of these capabilities have potential application to cognitive
systems, exploiting language models as a source of task knowledge, especially
for task learning, offers significant, near-term benefits. We introduce
language models and the various tasks to which they have been applied and then
review methods of knowledge extraction from language models. The resulting
analysis outlines both the challenges and opportunities for using language
models as a new knowledge source for cognitive systems. It also identifies
possible ways to improve knowledge extraction from language models using the
capabilities provided by cognitive systems. Central to success will be the
ability of a cognitive agent to itself learn an abstract model of the knowledge
implicit in the LM as well as methods to extract high-quality knowledge
effectively and efficiently. To illustrate, we introduce a hypothetical robot
agent and describe how language models could extend its task knowledge and
improve its performance and the kinds of knowledge and methods the agent can
use to exploit the knowledge within a language model.
