  Predicting human performance in interaction tasks allows designers or
developers to understand the expected performance of a target interface without
actually testing it with real users. In this work, we present a deep neural net
to model and predict human performance in performing a sequence of UI tasks. In
particular, we focus on a dominant class of tasks, i.e., target selection from
a vertical list or menu. We experimented with our deep neural net using a
public dataset collected from a desktop laboratory environment and a dataset
collected from hundreds of touchscreen smartphone users via crowdsourcing. Our
model significantly outperformed previous methods on these datasets.
Importantly, our method, as a deep model, can easily incorporate additional UI
attributes such as visual appearance and content semantics without changing
model architectures. By understanding about how a deep learning model learns
from human behaviors, our approach can be seen as a vehicle to discover new
patterns about human behaviors to advance analytical modeling.
