  Modern embedding-based metrics for evaluation of generated text generally
fall into one of two paradigms: discriminative metrics that are trained to
directly predict which outputs are of higher quality according to supervised
human annotations, and generative metrics that are trained to evaluate text
based on the probabilities of a generative model. Both have their advantages;
discriminative metrics are able to directly optimize for the problem of
distinguishing between good and bad outputs, while generative metrics can be
trained using abundant raw text. In this paper, we present a framework that
combines the best of both worlds, using both supervised and unsupervised
signals from whatever data we have available. We operationalize this idea by
training T5Score, a metric that uses these training signals with mT5 as the
backbone. We perform an extensive empirical comparison with other existing
metrics on 5 datasets, 19 languages and 280 systems, demonstrating the utility
of our method. Experimental results show that: T5Score achieves the best
performance on all datasets against existing top-scoring metrics at the segment
level. We release our code and models at https://github.com/qinyiwei/T5Score.
