  With the continuous popularity of deep learning and representation learning,
fast vector search becomes a vital task in various ranking/retrieval based
applications, say recommendation, ads ranking and question answering. Neural
network based ranking is widely adopted due to its powerful capacity in
modeling complex relationships, such as between users and items, questions and
answers. However, it is usually exploited in offline or re-ranking manners for
it is time-consuming in computations. Online neural network ranking--so called
fast neural ranking--is considered challenging because neural network measures
are usually non-convex and asymmetric. Traditional Approximate Nearest Neighbor
(ANN) search which usually focuses on metric ranking measures, is not
applicable to these advanced measures.
  In this paper, we introduce a novel graph searching framework to accelerate
the searching in the fast neural ranking problem. The proposed graph searching
algorithm is bi-level: we first construct a probable candidate set; then we
only evaluate the neural network measure over the probable candidate set
instead of evaluating the neural network over all neighbors. Specifically, we
propose a gradient-based algorithm that approximates the rank of the neural
network matching score to construct the probable candidate set; and we present
an angle-based heuristic procedure to adaptively identify the proper size of
the probable candidate set. Empirical results on public data confirm the
effectiveness of our proposed algorithms.
