  The increased presence of large language models (LLMs) in educational
settings has ignited debates concerning negative repercussions, including
overreliance and inadequate task reflection. Our work advocates moderated usage
of such models, designed in a way that supports students and encourages
critical thinking. We developed two moderated interaction methods with ChatGPT:
hint-based assistance and presenting multiple answer choices. In a study with
students (N=40) answering physics questions, we compared the effects of our
moderated models against two baseline settings: unmoderated ChatGPT access and
internet searches. We analyzed the interaction strategies and found that the
moderated versions exhibited less unreflected usage (e.g., copy \& paste)
compared to the unmoderated condition. However, neither ChatGPT-supported
condition could match the ratio of reflected usage present in internet
searches. Our research highlights the potential benefits of moderating language
models, showing a research direction toward designing effective AI-supported
educational strategies.
