  The SuperCLUE-Fin (SC-Fin) benchmark is a pioneering evaluation framework
tailored for Chinese-native financial large language models (FLMs). It assesses
FLMs across six financial application domains and twenty-five specialized
tasks, encompassing theoretical knowledge and practical applications such as
compliance, risk management, and investment analysis. Using multi-turn,
open-ended conversations that mimic real-life scenarios, SC-Fin measures models
on a range of criteria, including accurate financial understanding, logical
reasoning, clarity, computational efficiency, business acumen, risk perception,
and compliance with Chinese regulations.
  In a rigorous evaluation involving over a thousand questions, SC-Fin
identifies a performance hierarchy where domestic models like GLM-4 and
MoonShot-v1-128k outperform others with an A-grade, highlighting the potential
for further development in transforming theoretical knowledge into pragmatic
financial solutions. This benchmark serves as a critical tool for refining FLMs
in the Chinese context, directing improvements in financial knowledge
databases, standardizing financial interpretations, and promoting models that
prioritize compliance, risk management, and secure practices.
  We create a contextually relevant and comprehensive benchmark that drives the
development of AI in the Chinese financial sector. SC-Fin facilitates the
advancement and responsible deployment of FLMs, offering valuable insights for
enhancing model performance and usability for both individual and institutional
users in the Chinese market..~\footnote{Our benchmark can be found at
\url{https://www.CLUEbenchmarks.com}}.
