  The evaluation of large language models is an essential task in the field of
language understanding and generation. As language models continue to advance,
the need for effective benchmarks to assess their performance has become
imperative. In the context of Traditional Chinese, there is a scarcity of
comprehensive and diverse benchmarks to evaluate the capabilities of language
models, despite the existence of certain benchmarks such as DRCD, TTQA, CMDQA,
and FGC dataset. To address this gap, we propose a novel set of benchmarks that
leverage existing English datasets and are tailored to evaluate language models
in Traditional Chinese. These benchmarks encompass a wide range of tasks,
including contextual question-answering, summarization, classification, and
table understanding. The proposed benchmarks offer a comprehensive evaluation
framework, enabling the assessment of language models' capabilities across
different tasks. In this paper, we evaluate the performance of GPT-3.5,
Taiwan-LLaMa-v1.0, and Model 7-C, our proprietary model, on these benchmarks.
The evaluation results highlight that our model, Model 7-C, achieves
performance comparable to GPT-3.5 with respect to a part of the evaluated
capabilities. In an effort to advance the evaluation of language models in
Traditional Chinese and stimulate further research in this field, we have
open-sourced our benchmark and opened the model for trial.
