  Over the past two decades, dialogue modeling has made significant strides,
moving from simple rule-based responses to personalized and persuasive response
generation. However, despite these advancements, the objective functions and
evaluation metrics for dialogue generation have remained stagnant. These
lexical-based metrics, e.g., cross-entropy and BLEU, have two key limitations:
(a) word-to-word matching without semantic consideration: It assigns the same
credit for failure to generate "nice" and "rice" for "good", (b) missing
context attribute for evaluating the generated response: Even if a generated
response is relevant to the ongoing dialogue context, it may still be penalized
for not matching the gold utterance provided in the corpus. In this paper, we
first investigate these limitations comprehensively and propose a new loss
function called Semantic Infused Contextualized diaLogue (SemTextualLogue) loss
function. We also formulate an evaluation metric called Dialuation,
incorporating both context and semantic relevance. We experimented with both
non-pretrained and pre-trained models on two dialogue corpora, encompassing
task-oriented and open-domain scenarios. We found that the dialogue generation
models trained with SemTextualLogueloss attained superior performance compared
to the traditional cross-entropy loss function. The findings establish that the
effective training of a dialogue generation model hinges significantly on
incorporating semantics and context. This pattern is also mirrored in the
introduced Dialuation metric, where the consideration of both context and
semantics correlates more strongly with human evaluation compared to
traditional metrics.
