  The right response to someone who says "get ready for a party" is deeply
influenced by meaning and context. For a smart home assistant (e.g., Google
Home), the ideal response might be to survey the available devices in the home
and change their state to create a festive atmosphere. Current practical
systems cannot service such requests since they require the ability to (1)
infer meaning behind an abstract statement and (2) map that inference to a
concrete course of action appropriate for the context (e.g., changing the
settings of specific devices). In this paper, we leverage the observation that
recent task-agnostic large language models (LLMs) like GPT-3 embody a vast
amount of cross-domain, sometimes unpredictable contextual knowledge that
existing rule-based home assistant systems lack, which can make them powerful
tools for inferring user intent and generating appropriate context-dependent
responses during smart home interactions. We first explore the feasibility of a
system that places an LLM at the center of command inference and action
planning, showing that LLMs have the capacity to infer intent behind vague,
context-dependent commands like "get ready for a party" and respond with
concrete, machine-parseable instructions that can be used to control smart
devices. We furthermore demonstrate a proof-of-concept implementation that puts
an LLM in control of real devices, showing its ability to infer intent and
change device state appropriately with no fine-tuning or task-specific
training. Our work hints at the promise of LLM-driven systems for
context-awareness in smart environments, motivating future research in this
area.
