  Automatic open-domain dialogue evaluation has attracted increasing attention.
Trainable evaluation metrics are commonly trained with true positive and
randomly selected negative responses, resulting in a tendency for them to
assign a higher score to the responses that share higher content similarity
with a given context. However, adversarial negative responses possess high
content similarity with the contexts whilst being semantically different.
Therefore, existing evaluation metrics are not robust enough to evaluate such
responses, resulting in low correlations with human judgments. While recent
studies have shown some efficacy in utilizing Large Language Models (LLMs) for
open-domain dialogue evaluation, they still encounter challenges in effectively
handling adversarial negative examples. In this paper, we propose a simple yet
effective framework for open-domain dialogue evaluation, which combines
domain-specific language models (SLMs) with LLMs. The SLMs can explicitly
incorporate Abstract Meaning Representation (AMR) graph information of the
dialogue through a gating mechanism for enhanced semantic representation
learning. The evaluation result of SLMs and AMR graph information are plugged
into the prompt of LLM, for the enhanced in-context learning performance.
Experimental results on open-domain dialogue evaluation tasks demonstrate the
superiority of our method compared to a wide range of state-of-the-art
baselines, especially in discriminating adversarial negative responses. Our
code is available at https://github.com/Bernard-Yang/SIMAMR.
