  This study explores the application of Large Language Models (LLMs),
specifically GPT-4, in the analysis of classroom dialogue, a crucial research
task for both teaching diagnosis and quality improvement. Recognizing the
knowledge-intensive and labor-intensive nature of traditional qualitative
methods in educational research, this study investigates the potential of LLM
to streamline and enhance the analysis process. The study involves datasets
from a middle school, encompassing classroom dialogues across mathematics and
Chinese classes. These dialogues were manually coded by educational experts and
then analyzed using a customised GPT-4 model. This study focuses on comparing
manual annotations with the outputs of GPT-4 to evaluate its efficacy in
analyzing educational dialogues. Time efficiency, inter-coder agreement, and
inter-coder reliability between human coders and GPT-4 are evaluated. Results
indicate substantial time savings with GPT-4, and a high degree of consistency
in coding between the model and human coders, with some discrepancies in
specific codes. These findings highlight the strong potential of LLM in
teaching evaluation and facilitation.
