  Many test coverage metrics have been proposed to measure the Deep Neural
Network (DNN) testing effectiveness, including structural coverage and
non-structural coverage. These test coverage metrics are proposed based on the
fundamental assumption: they are correlated with test effectiveness. However,
the fundamental assumption is still not validated sufficiently and reasonably,
which brings question on the usefulness of DNN test coverage. This paper
conducted a revisiting study on the existing DNN test coverage from the test
effectiveness perspective, to effectively validate the fundamental assumption.
Here, we carefully considered the diversity of subjects, three test
effectiveness criteria, and both typical and state-of-the-art test coverage
metrics. Different from all the existing studies that deliver negative
conclusions on the usefulness of existing DNN test coverage, we identified some
positive conclusions on their usefulness from the test effectiveness
perspective. In particular, we found the complementary relationship between
structural and non-structural coverage and identified the practical usage
scenarios and promising research directions for these existing test coverage
metrics.
