  Recent advancements in Large Vision Language Models (LVLMs) have
revolutionized how machines understand and generate textual responses based on
visual inputs. Despite their impressive capabilities, they often produce
"hallucinatory" outputs that do not accurately reflect the visual information,
posing challenges in reliability and trustworthiness. Current methods such as
contrastive decoding have made strides in addressing these issues by
contrasting the original probability distribution of generated tokens with
distorted counterparts; yet, generating visually-faithful outputs remains a
challenge. In this work, we shift our focus to the opposite: What could serve
as a complementary enhancement to the original probability distribution? We
propose a simple, training-free method termed RITUAL to enhance robustness
against hallucinations in LVLMs. Our approach employs random image
transformations as complements to the original probability distribution, aiming
to mitigate the likelihood of hallucinatory visual explanations by enriching
the model's exposure to varied visual scenarios. Our empirical results show
that while the isolated use of transformed images initially degrades
performance, strategic implementation of these transformations can indeed serve
as effective complements. Notably, our method is compatible with current
contrastive decoding methods and does not require external models or costly
self-feedback mechanisms, making it a practical addition. In experiments,
RITUAL significantly outperforms existing contrastive decoding methods across
several object hallucination benchmarks, including POPE, CHAIR, and MME.
