  Automatically generating test suites is intrinsically a multi-objective
problem, as any of the testing targets (e.g, statements to execute or mutants
to kill) is an objective on its own. Test suite generation has peculiarities
that are quite different from other more regular optimisation problems. For
example, given an existing test suite, one can add more tests to cover the
remaining objectives. One would like the smallest number of small tests to
cover as many objectives as possible, but that is a secondary goal compared to
covering those targets in the first place. Furthermore, the amount of
objectives in software testing can quickly become unmanageable, in the order of
(tens/hundreds of) thousands, especially for system testing of industrial size
systems. Traditional multi-objective optimisation algorithms can already start
to struggle with just four or five objectives to optimize. To overcome these
issues, different techniques have been proposed, like for example the Whole
Test Suite (WTS) approach and the Many-Objective Sorting Algorithm (MOSA).
However, those techniques might not scale well to very large numbers of
objectives and limited search budgets (a typical case in system testing). In
this paper, we propose a novel algorithm, called Many Independent Objective
(MIO) algorithm. This algorithm is designed and tailored based on the specific
properties of test suite generation. An empirical study, on a set of artificial
and actual software, shows that the MIO algorithm can achieve higher coverage
compared to WTS and MOSA, as it can better exploit the peculiarities of test
suite generation.
