  Ensuring factual consistency is crucial for natural language generation
tasks, particularly in abstractive summarization, where preserving the
integrity of information is paramount. Prior works on evaluating factual
consistency of summarization often take the entailment-based approaches that
first generate perturbed (factual inconsistent) summaries and then train a
classifier on the generated data to detect the factually inconsistencies during
testing time. However, previous approaches generating perturbed summaries are
either of low coherence or lack error-type coverage. To address these issues,
we propose AMRFact, a framework that generates perturbed summaries using
Abstract Meaning Representations (AMRs). Our approach parses factually
consistent summaries into AMR graphs and injects controlled factual
inconsistencies to create negative examples, allowing for coherent factually
inconsistent summaries to be generated with high error-type coverage.
Additionally, we present a data selection module NegFilter based on natural
language inference and BARTScore to ensure the quality of the generated
negative samples. Experimental results demonstrate our approach significantly
outperforms previous systems on the AggreFact-SOTA benchmark, showcasing its
efficacy in evaluating factuality of abstractive summarization.
