  Abstractive summarization is the process of generating a summary given a
document as input. Although significant progress has been made, the factual
inconsistency between the document and the generated summary still limits its
practical applications. Previous work found that the probabilities assigned by
the generation model reflect its preferences for the generated summary,
including the preference for factual consistency, and the preference for the
language or knowledge prior as well. To separate the preference for factual
consistency, we propose an unsupervised framework named CoP by controlling the
preference of the generation model with the help of prompt. More specifically,
the framework performs an extra inference step in which a text prompt is
introduced as an additional input. In this way, another preference is described
by the generation probability of this extra inference process. The difference
between the above two preferences, i.e. the difference between the
probabilities, could be used as measurements for detecting factual
inconsistencies. Interestingly, we found that with the properly designed
prompt, our framework could evaluate specific preferences and serve as
measurements for fine-grained categories of inconsistency, such as
entity-related inconsistency, coreference-related inconsistency, etc. Moreover,
our framework could also be extended to the supervised setting to learn better
prompt from the labeled data as well. Experiments show that our framework
achieves new SOTA results on three factual inconsistency detection tasks.
