  Query expansion is an effective approach for mitigating vocabulary mismatch
between queries and documents in information retrieval. One recent line of
research uses language models to generate query-related contexts for expansion.
Along this line, we argue that expansion terms from these contexts should
balance two key aspects: diversity and relevance. The obvious way to increase
diversity is to sample multiple contexts from the language model. However, this
comes at the cost of relevance, because there is a well-known tendency of
models to hallucinate incorrect or irrelevant contexts. To balance these two
considerations, we propose a combination of an effective filtering strategy and
fusion of the retrieved documents based on the generation probability of each
context. Our lexical matching based approach achieves a similar top-5/top-20
retrieval accuracy and higher top-100 accuracy compared with the
well-established dense retrieval model DPR, while reducing the index size by
more than 96%. For end-to-end QA, the reader model also benefits from our
method and achieves the highest Exact-Match score against several competitive
baselines.
