  Recently, neural approaches to spoken content retrieval have become popular.
However, they tend to be restricted in their vocabulary or in their ability to
deal with imbalanced test settings. These restrictions limit their
applicability in keyword search, where the set of queries is not known
beforehand, and where the system should return not just whether an utterance
contains a query but the exact location of any such occurrences. In this work,
we propose a model directly optimized for keyword search. The model takes a
query and an utterance as input and returns a sequence of probabilities for
each frame of the utterance of the query having occurred in that frame.
Experiments show that the proposed model not only outperforms similar
end-to-end models on a task where the ratio of positive and negative trials is
artificially balanced, but it is also able to deal with the far more
challenging task of keyword search with its inherent imbalance. Furthermore,
using our system to rescore the outputs an LVCSR-based keyword search system
leads to significant improvements on the latter.
