  A core interest in building Artificial Intelligence (AI) agents is to let
them interact with and assist humans. One example is Dynamic Search (DS), which
models the process that a human works with a search engine agent to accomplish
a complex and goal-oriented task. Early DS agents using Reinforcement Learning
(RL) have only achieved limited success for (1) their lack of direct control
over which documents to return and (2) the difficulty to recover from wrong
search trajectories. In this paper, we present a novel corpus-level end-to-end
exploration (CE3) method to address these issues. In our method, an entire text
corpus is compressed into a global low-dimensional representation, which
enables the agent to gain access to the full state and action spaces, including
the under-explored areas. We also propose a new form of retrieval function,
whose linear approximation allows end-to-end manipulation of documents.
Experiments on the Text REtrieval Conference (TREC) Dynamic Domain (DD) Track
show that CE3 outperforms the state-of-the-art DS systems.
