  Search engines operate under a strict time constraint as a fast response is
paramount to user satisfaction. Thus, neural re-ranking models have a limited
time-budget to re-rank documents. Given the same amount of time, a faster
re-ranking model can incorporate more documents than a less efficient one,
leading to a higher effectiveness. To utilize this property, we propose TK
(Transformer-Kernel): a neural re-ranking model for ad-hoc search using an
efficient contextualization mechanism. TK employs a very small number of
Transformer layers (up to three) to contextualize query and document word
embeddings. To score individual term interactions, we use a document-length
enhanced kernel-pooling, which enables users to gain insight into the model. TK
offers an optimal ratio between effectiveness and efficiency: under realistic
time constraints (max. 200 ms per query) TK achieves the highest effectiveness
in comparison to BERT and other re-ranking models. We demonstrate this on three
large-scale ranking collections: MSMARCO-Passage, MSMARCO-Document, and TREC
CAR. In addition, to gain insight into TK, we perform a clustered query
analysis of TK's results, highlighting its strengths and weaknesses on queries
with different types of information need and we show how to interpret the cause
of ranking differences of two documents by comparing their internal scores.
