  An emotional support conversation system aims to alleviate users' emotional
distress and assist them in addressing their challenges. To generate supportive
responses, it is critical to consider multiple factors such as empathy, support
strategies, and response coherence, as established in prior methods.
Nonetheless, previous models occasionally generate unhelpful responses, which
intend to provide support but display counterproductive effects. According to
psychology and communication theories, poor performance in just one
contributing factor might cause a response to be unhelpful. From the model
training perspective, since these models have not been exposed to unhelpful
responses during their training phase, they are unable to distinguish if the
tokens they generate might result in unhelpful responses during inference. To
address this issue, we introduce a novel model-agnostic framework named
mitigating unhelpfulness with multifaceted AI feedback for emotional support
(Muffin). Specifically, Muffin employs a multifaceted AI feedback module to
assess the helpfulness of responses generated by a specific model with
consideration of multiple factors. Using contrastive learning, it then reduces
the likelihood of the model generating unhelpful responses compared to the
helpful ones. Experimental results demonstrate that Muffin effectively
mitigates the generation of unhelpful responses while slightly increasing
response fluency and relevance.
