  Integrating automatic speech scoring/assessment systems has become a critical
aspect of second-language speaking education. With self-supervised learning
advancements, end-to-end speech scoring approaches have exhibited promising
results. However, this study highlights the significant decrease in the
performance of speech scoring systems in new question contexts, thereby
identifying this as a cold start problem in terms of items. With the finding of
cold-start phenomena, this paper seeks to alleviate the problem by following
methods: 1) prompt embeddings, 2) question context embeddings using BERT or
CLIP models, and 3) choice of the pretrained acoustic model. Experiments are
conducted on TOEIC speaking test datasets collected from
English-as-a-second-language (ESL) learners rated by professional TOEIC
speaking evaluators. The results demonstrate that the proposed framework not
only exhibits robustness in a cold-start environment but also outperforms the
baselines for known content.
