  Despite some recent advances, automatic text summarization remains
unreliable, elusive, and of limited practical use in applications. Two main
problems with current summarization methods are well known: evaluation and
factual consistency. To address these issues, we propose a new automatic
reference-less text summarization evaluation system that can measure the
quality of any text summarization model with a set of generated facts based on
factual consistency, comprehensiveness, and compression rate. As far as we
know, our evaluation system is the first system that measures the overarching
quality of the text summarization models based on factuality, information
coverage, and compression rate.
