  Randomized A/B comparisons of alternative pedagogical strategies or other
course improvements could provide useful empirical evidence for instructor
decision-making. However, traditional experiments do not provide a
straightforward pathway to rapidly utilize data, increasing the chances that
students in an experiment experience the best conditions. Drawing inspiration
from the use of machine learning and experimentation in product development at
leading technology companies, we explore how adaptive experimentation might aid
continuous course improvement. In adaptive experiments, data is analyzed and
utilized as different conditions are deployed to students. This can be achieved
using machine learning algorithms to identify which actions are more beneficial
in improving students' learning experiences and outcomes. These algorithms can
then dynamically deploy the most effective conditions in subsequent
interactions with students, resulting in better support for students' needs. We
illustrate this approach with a case study that provides a side-by-side
comparison of traditional and adaptive experiments on adding self-explanation
prompts in online homework problems in a CS1 course. This work paves the way
for exploring the importance of adaptive experiments in bridging research and
practice to achieve continuous improvement in educational settings.
