  Large language models achieve state-of-the-art performance on sequence
generation evaluation, but typically have a large number of parameters. This is
a computational challenge as presented by applying their evaluation capability
at scale. To overcome the challenge, in this paper, we propose \textbf{ECT}, an
\textbf{e}valuation \textbf{c}apability \textbf{t}ransfer method, to transfer
the evaluation capability from LLMs to relatively lightweight language models.
Based on the proposed ECT, we learn various evaluation models from ChatGPT, and
employ them as reward models to improve sequence generation models via
reinforcement learning and reranking approaches. Experimental results on
machine translation, text style transfer, and summarization tasks demonstrate
the effectiveness of our ECT. Notably, applying the learned evaluation models
to sequence generation models results in better generated sequences as
evaluated by commonly used metrics and ChatGPT.
