  We conducted a systematic literature review on automated grading and feedback
tools for programming education.
  We analysed 121 research papers from 2017 to 2021 inclusive and categorised
them based on skills assessed, approach, language paradigm, degree of
automation and evaluation techniques.
  Most papers assess the correctness of assignments in object-oriented
languages.
  Typically, these tools use a dynamic technique, primarily unit testing, to
provide grades and feedback to the students or static analysis techniques to
compare a submission with a reference solution or with a set of correct student
submissions.
  However, these techniques' feedback is often limited to whether the unit
tests have passed or failed, the expected and actual output, or how they differ
from the reference solution.
  Furthermore, few tools assess the maintainability, readability or
documentation of the source code, with most using static analysis techniques,
such as code quality metrics, in conjunction with grading correctness.
  Additionally, we found that most tools offered fully automated assessment to
allow for near-instantaneous feedback and multiple resubmissions, which can
increase student satisfaction and provide them with more opportunities to
succeed.
  In terms of techniques used to evaluate the tools' performance, most papers
primarily use student surveys or compare the automatic assessment tools to
grades or feedback provided by human graders.
  However, because the evaluation dataset is frequently unavailable, it is more
difficult to reproduce results and compare tools to a collection of common
assignments.
