  In this paper, we explore the potential of Large Language Models (LLMs) with
assertions to mitigate imbalances in educational datasets. Traditional models
often fall short in such contexts, particularly due to the complexity and
nuanced nature of the data. This issue is especially prominent in the education
sector, where cognitive engagement levels among students show significant
variation in their open responses. To test our hypothesis, we utilized an
existing technology for assertion-based prompt engineering through an
'Iterative - ICL PE Design Process' comparing traditional Machine Learning (ML)
models against LLMs augmented with assertions (N=135). Further, we conduct a
sensitivity analysis on a subset (n=27), examining the variance in model
performance concerning classification metrics and cognitive engagement levels
in each iteration. Our findings reveal that LLMs with assertions significantly
outperform traditional ML models, particularly in cognitive engagement levels
with minority representation, registering up to a 32% increase in F1-score.
Additionally, our sensitivity study indicates that incorporating targeted
assertions into the LLM tested on the subset enhances its performance by
11.94%. This improvement primarily addresses errors stemming from the model's
limitations in understanding context and resolving lexical ambiguities in
student responses.
