  The fragmentation problem has extended from Android to different platforms,
such as iOS, mobile web, and even mini-programs within some applications (app).
In such a situation, recording and replaying test scripts is a popular
automated mobile app testing approaches. But such approach encounters severe
problems when crossing platforms. Different versions of the same app need to be
developed to support different platforms relying on different platform
supports. Therefore, mobile app developers need to develop and maintain test
scripts for multiple platforms aimed at completely the same test requirements,
greatly increasing testing costs. However, we discover that developers adopt
highly similar user interface layouts for versions of the same app on different
platforms. Such a phenomenon inspires us to replay test scripts from the
perspective of similar UI layouts. We propose an image-driven mobile app
testing framework, utilizing Widget Feature Matching and Layout
Characterization Matching. We use computer vision technologies to perform UI
feature comparison and layout hierarchy extraction on app screenshots to obtain
UI structures with rich contextual information, including coordinates, relative
relationship, etc. Based on acquired UI structures, we can form a
platform-independent test script, and then locate the target widgets under
test. Thus, the proposed framework non-intrusively replays test scripts
according to a novel platform-independent test script model. We also design and
implement a tool named LIT to devote the proposed framework into practice,
based on which, we conduct an empirical study to evaluate the effectiveness and
usability of the proposed testing framework. Results show that the overall
replay accuracy reaches around 63.39% on Android (14% improvement over
state-of-the-art approaches) and 21.83% on iOS (98% improvement over
state-of-the-art approaches).
