  Multilingual LLMs often have knowledge disparities across languages, with
larger gaps in under-resourced languages. Teaching LLMs to abstain in the face
of knowledge gaps is thus a promising strategy to mitigate hallucinations in
multilingual settings. However, previous studies on LLM abstention primarily
focus on English; we find that directly applying existing solutions beyond
English results in up to 20.5% performance gaps between high and low-resource
languages, potentially due to LLMs' drop in calibration and reasoning beyond a
few resource-rich languages. To this end, we propose strategies to enhance LLM
abstention by learning from multilingual feedback, where LLMs self-reflect on
proposed answers in one language by generating multiple feedback items in
related languages: we show that this helps identifying the knowledge gaps
across diverse languages, cultures, and communities. Extensive experiments
demonstrate that our multilingual feedback approach outperforms various strong
baselines, achieving up to 9.2% improvement for low-resource languages across
three black-box and open models on three datasets, featuring open-book,
closed-book, and commonsense QA. Further analysis reveals that multilingual
feedback is both an effective and a more equitable abstain strategy to serve
diverse language speakers, and cultural factors have great impact on language
selection and LLM abstention behavior, highlighting future directions for
multilingual and multi-cultural reliable language modeling.
