  Adaptive user interfaces (UIs) automatically change an interface to better
support users' tasks. Recently, machine learning techniques have enabled the
transition to more powerful and complex adaptive UIs. However, a core challenge
for adaptive user interfaces is the reliance on high-quality user data that has
to be collected offline for each task. We formulate UI adaptation as a
multi-agent reinforcement learning problem to overcome this challenge. In our
formulation, a user agent mimics a real user and learns to interact with a UI.
Simultaneously, an interface agent learns UI adaptations to maximize the user
agent's performance. The interface agent learns the task structure from the
user agent's behavior and, based on that, can support the user agent in
completing its task. Our method produces adaptation policies that are learned
in simulation only and, therefore, does not need real user data. Our
experiments show that learned policies generalize to real users and achieve on
par performance with data-driven supervised learning baselines.
