  This research investigates the effect of prompt design on dialogue evaluation
using large language models (LLMs). While LLMs are increasingly used for
scoring various inputs, creating effective prompts for dialogue evaluation
remains challenging due to model sensitivity and subjectivity in dialogue
assessments. Our study experimented with different prompt structures, altering
the sequence of output instructions and including explanatory reasons. We found
that the order of presenting reasons and scores significantly influences LLMs'
scoring, with a "reason-first" approach yielding more comprehensive
evaluations. This insight is crucial for enhancing the accuracy and consistency
of LLM-based evaluations.
