  We propose a method to efficiently learn diverse strategies in reinforcement
learning for query reformulation in the tasks of document retrieval and
question answering. In the proposed framework an agent consists of multiple
specialized sub-agents and a meta-agent that learns to aggregate the answers
from sub-agents to produce a final answer. Sub-agents are trained on disjoint
partitions of the training data, while the meta-agent is trained on the full
training set. Our method makes learning faster, because it is highly
parallelizable, and has better generalization performance than strong
baselines, such as an ensemble of agents trained on the full data. We show that
the improved performance is due to the increased diversity of reformulation
strategies.
