  An Android-based smart Television (TV) must reliably run its applications in
an embedded program environment under diverse hardware resource conditions.
Owing to the diverse hardware components used to build numerous TV models, TV
simulators are usually not high enough in fidelity to simulate various TV
models, and thus are only regarded as unreliable alternatives when stress
testing such applications. Therefore, even though stress testing on real TV
sets is tedious, it is the de facto approach to ensure the reliability of these
applications in the industry. In this paper, we study to what extent stress
testing of smart TV applications can be fully automated in the industrial
environments. To the best of our knowledge, no previous work has addressed this
important question. We summarize the find-ings collected from 10 industrial
test engineers to have tested 20 such TV applications in a real production
environment. Our study shows that the industry required test automation
supports on high-level GUI object controls and status checking, setup of
resource conditions and the interplay between the two. With such supports, 87%
of the industrial test specifications of one TV model can be fully automated
and 71.4% of them were found to be fully reusable to test a subsequent TV model
with major up-grades of hardware, operating system and application. It
repre-sents a significant improvement with margins of 28% and 38%,
respectively, compared to stress testing without such supports.
