  Writing and maintaining UI tests for mobile apps is a time-consuming and
tedious task. While decades of research have produced automated approaches for
UI test generation, these approaches typically focus on testing for crashes or
maximizing code coverage. By contrast, recent research has shown that
developers prefer usage-based tests, which center around specific uses of app
features, to help support activities such as regression testing. Very few
existing techniques support the generation of such tests, as doing so requires
automating the difficult task of understanding the semantics of UI screens and
user inputs. In this paper, we introduce Avgust, which automates key steps of
generating usage-based tests. Avgust uses neural models for image understanding
to process video recordings of app uses to synthesize an app-agnostic
state-machine encoding of those uses. Then, Avgust uses this encoding to
synthesize test cases for a new target app. We evaluate Avgust on 374 videos of
common uses of 18 popular apps and show that 69% of the tests Avgust generates
successfully execute the desired usage, and that Avgust's classifiers
outperform the state of the art.
