  Developing interactive software, such as websites or games, is a particularly
engaging way to learn computer science. However, teaching and giving feedback
on such software is time-consuming -- standard approaches require instructors
to manually grade student-implemented interactive programs. As a result, online
platforms that serve millions, like Code.org, are unable to provide any
feedback on assignments for implementing interactive programs, which critically
hinders students' ability to learn. One approach toward automatic grading is to
learn an agent that interacts with a student's program and explores states
indicative of errors via reinforcement learning. However, existing work on this
approach only provides binary feedback of whether a program is correct or not,
while students require finer-grained feedback on the specific errors in their
programs to understand their mistakes. In this work, we show that exploring to
discover errors can be cast as a meta-exploration problem. This enables us to
construct a principled objective for discovering errors and an algorithm for
optimizing this objective, which provides fine-grained feedback. We evaluate
our approach on a set of over 700K real anonymized student programs from a
Code.org interactive assignment. Our approach provides feedback with 94.3%
accuracy, improving over existing approaches by 17.7% and coming within 1.5% of
human-level accuracy. Project web page: https://ezliu.github.io/dreamgrader.
