  Automated test case generation is an effective technique to yield
high-coverage test suites. While the majority of research effort has been
devoted to satisfying coverage criteria, a recent trend emerged towards
optimizing other non-coverage aspects. In this regard, runtime and memory usage
are two essential dimensions: less expensive tests reduce the resource demands
for the generation process and later regression testing phases. This study
shows that performance-aware test case generation requires solving two main
challenges: providing a good approximation of resource usage with minimal
overhead and avoiding detrimental effects on both final coverage and fault
detection effectiveness. To tackle these challenges, we conceived a set of
performance proxies -- inspired by previous work on performance testing -- that
provide a reasonable estimation of the test execution costs (i.e., runtime and
memory usage). Thus, we propose an adaptive strategy, called aDynaMOSA, which
leverages these proxies by extending DynaMOSA, a state-of-the-art evolutionary
algorithm in unit testing. Our empirical study -- involving 110 non-trivial
Java classes -- reveals that our adaptive approach generates test suite with
statistically significant improvements in runtime (-25%) and heap memory
consumption (-15%) compared to DynaMOSA. Additionally, aDynaMOSA has comparable
results to DynaMOSA over seven different coverage criteria and similar fault
detection effectiveness. Our empirical investigation also highlights that the
usage of performance proxies (i.e., without the adaptiveness) is not sufficient
to generate more performant test cases without compromising the overall
coverage.
