  This paper unveils CG-Eval, the first-ever comprehensive and automated
evaluation framework designed for assessing the generative capabilities of
large Chinese language models across a spectrum of academic disciplines.
CG-Eval stands out for its automated process, which critically assesses models
based on their proficiency in generating precise and contextually relevant
responses to a diverse array of questions within six key domains: Science and
Engineering, Humanities and Social Sciences, Mathematical Calculations, Medical
Practitioner Qualification Examination, Judicial Examination, and Certified
Public Accountant Examination. Alongside this, we introduce Gscore, an
innovative composite index developed from a weighted sum of multiple metrics.
Gscore uniquely automates the quality measurement of a model's text generation
against reference standards, providing a detailed and nuanced assessment of
model performance. This automation not only enhances the efficiency and
scalability of the evaluation process but also ensures objective and consistent
assessment across various models. The detailed test data and results,
highlighting the robust capabilities and comparative performance of the
evaluated models, are accessible at http://cgeval.besteasy.com/.
