  This paper presents a way of enhancing the reliability of Large Multi-modal
Models (LMMs) in addressing hallucination, where the models generate
cross-modal inconsistent responses. Without additional training, we propose
Counterfactual Inception, a novel method that implants counterfactual thinking
into LMMs using self-generated counterfactual keywords. Our method is grounded
in the concept of counterfactual thinking, a cognitive process where human
considers alternative realities, enabling more extensive context exploration.
Bridging the human cognition mechanism into LMMs, we aim for the models to
engage with and generate responses that span a wider contextual scene
understanding, mitigating hallucinatory outputs. We further introduce
Plausibility Verification Process (PVP), a simple yet robust keyword constraint
that effectively filters out sub-optimal keywords to enable the consistent
triggering of counterfactual thinking in the model responses. Comprehensive
analyses across various LMMs, including both open-source and proprietary
models, corroborate that counterfactual thinking significantly reduces
hallucination and helps to broaden contextual understanding based on true
visual clues.
