  The feedback provided by current testing education tools about the
deficiencies in a student's test suite either mimics industry code coverage
tools or lists specific instructor test cases that are missing from the
student's test suite. While useful in some sense, these types of feedback are
akin to revealing the solution to the problem, which can inadvertently
encourage students to pursue a trial-and-error approach to testing, rather than
using a more systematic approach that encourages learning. In addition to not
teaching students why their test suite is inadequate, this type of feedback may
motivate students to become dependent on the feedback rather than thinking for
themselves. To address this deficiency, there is an opportunity to investigate
alternative feedback mechanisms that include a positive reinforcement of
testing concepts. We argue that using an inquiry-based learning approach is
better than simply providing the answers. To facilitate this type of learning,
we present Testing Tutor, a web-based assignment submission platform that
supports different levels of testing pedagogy via a customizable feedback
engine. We evaluated the impact of the different types of feedback through an
empirical study in two sophomore-level courses. We use Testing Tutor to provide
students with different types of feedback, either traditional detailed code
coverage feedback or inquiry-based learning conceptual feedback, and compare
the effects. The results show that students that receive conceptual feedback
had higher code coverage (by different measures), fewer redundant test cases,
and higher programming grades than the students who receive traditional code
coverage feedback.
