  Large vision-language models (LVLMs) have shown promising performance on a
variety of vision-language tasks. However, they remain susceptible to
hallucinations, generating outputs misaligned with visual content or
instructions. While various mitigation strategies have been proposed, they
often neglect a key contributor to hallucinations: lack of fine-grained
reasoning supervision during training. Without intermediate reasoning steps,
models may establish superficial shortcuts between instructions and responses,
failing to internalize the inherent reasoning logic. To address this challenge,
we propose reflective instruction tuning, which integrates rationale learning
into visual instruction tuning. Unlike previous methods that learning from
responses only, our approach entails the model predicting rationales justifying
why responses are correct or incorrect. This fosters a deeper engagement with
the fine-grained reasoning underlying each response, thus enhancing the model's
reasoning proficiency. To facilitate this approach, we propose REVERIE, the
first large-scale instruction-tuning dataset with ReflEctiVE RatIonalE
annotations. REVERIE comprises 115k machine-generated reasoning instructions,
each meticulously annotated with a corresponding pair of correct and confusing
responses, alongside comprehensive rationales elucidating the justification
behind the correctness or erroneousness of each response. Experimental results
on multiple LVLM benchmarks reveal that reflective instruction tuning with the
REVERIE dataset yields noticeable performance gain over the baseline model,
demonstrating the effectiveness of reflecting from the rationales. Project page
is at https://zjr2000.github.io/projects/reverie.
