  In the era of rapid evolution of generative language models within the realm
of natural language processing, there is an imperative call to revisit and
reformulate evaluation methodologies, especially in the domain of aspect-based
sentiment analysis (ABSA). This paper addresses the emerging challenges
introduced by the generative paradigm, which has moderately blurred traditional
boundaries between understanding and generation tasks. Building upon prevailing
practices in the field, we analyze the advantages and shortcomings associated
with the prevalent ABSA evaluation paradigms. Through an in-depth examination,
supplemented by illustrative examples, we highlight the intricacies involved in
aligning generative outputs with other evaluative metrics, specifically those
derived from other tasks, including question answering. While we steer clear of
advocating for a singular and definitive metric, our contribution lies in
paving the path for a comprehensive guideline tailored for ABSA evaluations in
this generative paradigm. In this position paper, we aim to provide
practitioners with profound reflections, offering insights and directions that
can aid in navigating this evolving landscape, ensuring evaluations that are
both accurate and reflective of generative capabilities.
