  This paper presents first successful steps in designing search agents that
learn meta-strategies for iterative query refinement in information-seeking
tasks. Our approach uses machine reading to guide the selection of refinement
terms from aggregated search results. Agents are then empowered with simple but
effective search operators to exert fine-grained and transparent control over
queries and search results. We develop a novel way of generating synthetic
search sessions, which leverages the power of transformer-based language models
through (self-)supervised learning. We also present a reinforcement learning
agent with dynamically constrained actions that learns interactive search
strategies from scratch. Our search agents obtain retrieval and answer quality
performance comparable to recent neural methods, using only a traditional
term-based BM25 ranking function and interpretable discrete reranking and
filtering actions.
