  Large language models (LLMs) have demonstrated multilingual capabilities;
yet, they are mostly English-centric due to the imbalanced training corpora.
Existing works leverage this phenomenon to improve their multilingual
performances through translation, primarily on natural language processing
(NLP) tasks. This work extends the evaluation from NLP tasks to real user
queries and from English-centric LLMs to non-English-centric LLMs. While
translation into English can help improve the performance of multilingual NLP
tasks for English-centric LLMs, it may not be optimal for all scenarios. For
culture-related tasks that need deep language understanding, prompting in the
native language tends to be more promising as it better captures the nuances of
culture and language. Our experiments reveal varied behaviors among different
LLMs and tasks in the multilingual context. Therefore, we advocate for more
comprehensive multilingual evaluation and more efforts toward developing
multilingual LLMs beyond English-centric ones.
