  Computing theory analyzes abstract computational models to rigorously study
the computational difficulty of various problems. Introductory computing theory
can be challenging for undergraduate students, and the main goal of our
research is to help students learn these computational models. The most common
pedagogical tool for interacting with these models is the Java Formal Languages
and Automata Package (JFLAP). We developed a JFLAP server extension, which
accepts homework submissions from students, evaluates the submission as correct
or incorrect, and provides a witness string when the submission is incorrect.
Our extension currently provides witness feedback for deterministic finite
automata, nondeterministic finite automata, regular expressions, context-free
grammars, and pushdown automata.
  In Fall 2019, we ran a preliminary investigation on two sections (Control and
Study) of the required undergraduate course Introduction to Computer Science
Theory. The Study section used our extension for five targeted homework
questions, and the Control section solved and submitted these problems using
traditional means. Our results show that on these five questions, the Study
section performed better on average than the Control section. Moreover, the
Study section persisted in submitting attempts until correct, and from this
finding, our preliminary conclusion is that minimal (not detailed or
grade-based) witness feedback helps students to truly learn the concepts. We
describe the results that support this conclusion as well as a related
hypothesis conjecturing that with witness feedback and unlimited number of
submissions, partial credit is both unnecessary and ineffective.
