  This paper investigates the cross-lingual inconsistencies observed in Large
Language Models (LLMs), such as ChatGPT, Llama, and Baichuan, which have shown
exceptional performance in various Natural Language Processing (NLP) tasks.
Despite their successes, these models often exhibit significant inconsistencies
when processing the same concepts across different languages. This study
focuses on three primary questions: the existence of cross-lingual
inconsistencies in LLMs, the specific aspects in which these inconsistencies
manifest, and the correlation between cross-lingual consistency and
multilingual capabilities of LLMs.To address these questions, we propose an
innovative evaluation method for Cross-lingual Semantic Consistency (xSC) using
the LaBSE model. We further introduce metrics for Cross-lingual Accuracy
Consistency (xAC) and Cross-lingual Timeliness Consistency (xTC) to
comprehensively assess the models' performance regarding semantic, accuracy,
and timeliness inconsistencies. By harmonizing these metrics, we provide a
holistic measurement of LLMs' cross-lingual consistency. Our findings aim to
enhance the understanding and improvement of multilingual capabilities and
interpretability in LLMs, contributing to the development of more robust and
reliable multilingual language models.
