  The advances in language-based Artificial Intelligence (AI) technologies
applied to build educational applications can present AI for social-good
opportunities with a broader positive impact. Across many disciplines,
enhancing the quality of mathematics education is crucial in building critical
thinking and problem-solving skills at younger ages. Conversational AI systems
have started maturing to a point where they could play a significant role in
helping students learn fundamental math concepts. This work presents a
task-oriented Spoken Dialogue System (SDS) built to support play-based learning
of basic math concepts for early childhood education. The system has been
evaluated via real-world deployments at school while the students are
practicing early math concepts with multimodal interactions. We discuss our
efforts to improve the SDS pipeline built for math learning, for which we
explore utilizing MathBERT representations for potential enhancement to the
Natural Language Understanding (NLU) module. We perform an end-to-end
evaluation using real-world deployment outputs from the Automatic Speech
Recognition (ASR), Intent Recognition, and Dialogue Manager (DM) components to
understand how error propagation affects the overall performance in real-world
scenarios.
