  Large Language Models possess skills such as answering questions, writing
essays or solving programming exercises. Since these models are easily
accessible, researchers have investigated their capabilities and risks for
programming education. This work explores how LLMs can contribute to
programming education by supporting students with automated next-step hints. We
investigate prompt practices that lead to effective next-step hints and use
these insights to build our StAP-tutor. We evaluate this tutor by conducting an
experiment with students, and performing expert assessments. Our findings show
that most LLM-generated feedback messages describe one specific next step and
are personalised to the student's code and approach. However, the hints may
contain misleading information and lack sufficient detail when students
approach the end of the assignment. This work demonstrates the potential for
LLM-generated feedback, but further research is required to explore its
practical implementation.
