  Autocomplete suggestions are fundamental to modern text entry systems, with
applications in domains such as messaging and email composition. Typically,
autocomplete suggestions are generated from a language model with a confidence
threshold. However, this threshold does not directly take into account the
cognitive load imposed on the user by surfacing suggestions, such as the effort
to switch contexts from typing to reading the suggestion, and the time to
decide whether to accept the suggestion. In this paper, we study the problem of
improving inline autocomplete suggestions in text entry systems via a
sequential decision-making formulation, and use reinforcement learning to learn
suggestion policies through repeated interactions with a target user over time.
This formulation allows us to factor cognitive load into the objective of
training an autocomplete model, through a reward function based on text entry
speed. We acquired theoretical and experimental evidence that, under certain
objectives, the sequential decision-making formulation of the autocomplete
problem provides a better suggestion policy than myopic single-step reasoning.
However, aligning these objectives with real users requires further
exploration. In particular, we hypothesize that the objectives under which
sequential decision-making can improve autocomplete systems are not tailored
solely to text entry speed, but more broadly to metrics such as user
satisfaction and convenience.
