  While many hallucination detection techniques have been evaluated on English
text, their effectiveness in multilingual contexts remains unknown. This paper
assesses how well various factual hallucination detection metrics (lexical
metrics like ROUGE and Named Entity Overlap, and Natural Language Inference
(NLI)-based metrics) identify hallucinations in generated biographical
summaries across languages. We compare how well automatic metrics correlate to
each other and whether they agree with human judgments of factuality. Our
analysis reveals that while the lexical metrics are ineffective, NLI-based
metrics perform well, correlating with human annotations in many settings and
often outperforming supervised models. However, NLI metrics are still limited,
as they do not detect single-fact hallucinations well and fail for
lower-resource languages. Therefore, our findings highlight the gaps in
exisiting hallucination detection methods for non-English languages and
motivate future research to develop more robust multilingual detection methods
for LLM hallucinations.
