  Zero-shot visual question answering (VQA) is a challenging task that requires
reasoning across modalities. While some existing methods rely on a single
rationale within the Chain of Thoughts (CoT) framework, they may fall short of
capturing the complexity of the VQA problem. On the other hand, some other
methods that use multiple rationales may still suffer from low diversity, poor
modality alignment, and inefficient retrieval and fusion. In response to these
challenges, we propose \emph{Mixture of Rationales (MoR)}, a novel multi-modal
reasoning method that mixes multiple rationales for VQA. MoR uses a single
frozen Vision-and-Language Pre-trained Models (VLPM) model to {dynamically
generate, retrieve and fuse multi-modal thoughts}. We evaluate MoR on two
challenging VQA datasets, i.e. NLVR2 and OKVQA, with two representative
backbones OFA and VL-T5. MoR achieves a 12.43\% accuracy improvement on NLVR2,
and a 2.45\% accuracy improvement on OKVQA-S( the science and technology
category of OKVQA).
