  Evaluating automatically-generated text summaries is a challenging task.
While there have been many interesting approaches, they still fall short of
human evaluations. We present RISE, a new approach for evaluating summaries by
leveraging techniques from information retrieval. RISE is first trained as a
retrieval task using a dual-encoder retrieval setup, and can then be
subsequently utilized for evaluating a generated summary given an input
document, without gold reference summaries. RISE is especially well suited when
working on new datasets where one may not have reference summaries available
for evaluation. We conduct comprehensive experiments on the SummEval benchmark
(Fabbri et al., 2021) and the results show that RISE has higher correlation
with human evaluations compared to many past approaches to summarization
evaluation. Furthermore, RISE also demonstrates data-efficiency and
generalizability across languages.
