  GUI testing is significant in the SE community. Most existing frameworks are
intrusive and only support some specific platforms. With the development of
distinct scenarios, diverse embedded systems or customized operating systems on
different devices do not support existing intrusive GUI testing frameworks.
Some approaches adopt robotic arms to replace the interface invoking of mobile
apps under test and use computer vision technologies to identify GUI elements.
However, some challenges are unsolved. First, existing approaches assume that
GUI screens are fixed so that they cannot be adapted to diverse systems with
different screen conditions. Second, existing approaches use XY-plane robotic
arms, which cannot flexibly simulate testing operations. Third, existing
approaches ignore compatibility bugs and only focus on crash bugs. A more
practical approach is required for the non-intrusive scenario. We propose a
practical non-intrusive GUI testing framework with visual robotic arms.
RoboTest integrates novel GUI screen and widget detection algorithms, adaptive
to detecting screens of different sizes and then to extracting GUI widgets from
the detected screens. Then, a set of testing operations is applied with a 4-DOF
robotic arm, which effectively and flexibly simulates human testing operations.
During app exploration, RoboTest integrates the Principle of Proximity-guided
exploration strategy, choosing close widgets of the previous targets to reduce
robotic arm movement overhead and improve exploration efficiency. RoboTest can
effectively detect some compatibility bugs beyond crash bugs with a GUI
comparison on different devices of the same test operations. We evaluate
RoboTest with 20 mobile apps, with a case study on an embedded system. The
results show that RoboTest can effectively, efficiently, and generally explore
AUTs to find bugs and reduce exploration time overhead.
