  Evaluating large language models (LLMs) is fundamental, particularly in the
context of practical applications. Conventional evaluation methods, typically
designed primarily for LLM development, yield numerical scores that ignore the
user experience. Therefore, our study shifts the focus from model-centered to
human-centered evaluation in the context of AI-powered writing assistance
applications. Our proposed metric, termed ``Revision Distance,'' utilizes LLMs
to suggest revision edits that mimic the human writing process. It is
determined by counting the revision edits generated by LLMs. Benefiting from
the generated revision edit details, our metric can provide a self-explained
text evaluation result in a human-understandable manner beyond the
context-independent score. Our results show that for the easy-writing task,
``Revision Distance'' is consistent with established metrics (ROUGE,
Bert-score, and GPT-score), but offers more insightful, detailed feedback and
better distinguishes between texts. Moreover, in the context of challenging
academic writing tasks, our metric still delivers reliable evaluations where
other metrics tend to struggle. Furthermore, our metric also holds significant
potential for scenarios lacking reference texts.
