  Large language models, such as OpenAI's codex and Deepmind's AlphaCode, can
generate code to solve a variety of problems expressed in natural language.
This technology has already been commercialised in at least one widely-used
programming editor extension: GitHub Copilot.
  In this paper, we explore how programming with large language models
(LLM-assisted programming) is similar to, and differs from, prior
conceptualisations of programmer assistance. We draw upon publicly available
experience reports of LLM-assisted programming, as well as prior usability and
design studies. We find that while LLM-assisted programming shares some
properties of compilation, pair programming, and programming via search and
reuse, there are fundamental differences both in the technical possibilities as
well as the practical experience. Thus, LLM-assisted programming ought to be
viewed as a new way of programming with its own distinct properties and
challenges.
  Finally, we draw upon observations from a user study in which non-expert end
user programmers use LLM-assisted tools for solving data tasks in spreadsheets.
We discuss the issues that might arise, and open research challenges, in
applying large language models to end-user programming, particularly with users
who have little or no programming expertise.
