  With the advance of large language models (LLMs), the research field of LLM
applications becomes more and more popular and the idea of constructing
pipelines to accomplish complex tasks by stacking LLM API calls come true.
However, this kind of methods face two limitations: narrow information coverage
and low fault tolerance. In this work, we propose a novel method called ALLIES.
Given an input query, ALLIES leverages LLMs to iteratively generate new queries
related to the original query, enabling an iterative reasoning process. By
iteratively refining and expanding the scope of the original query, ALLIES
captures and utilizes hidden knowledge that may not be directly obtainable
through retrieval. We take zero-shot open-domain question answering (ODQA) as
an application scene and evaluate ALLIES on the widely-used benchmarks, such as
NQ, WebQ and TriviaQA. The experimental results demonstrate that ALLIES
significantly outperforms other zero-shot baselines, indicating its
effectiveness in tackling those challenges. Our code is available in
https://github.com/microsoft/SimXNS/tree/main/ALLIES.
