  In this paper we address the explainability of web search engines. We propose
two explainable elements on the search engine result page: a visualization of
query term weights and a visualization of passage relevance. The idea is that
search engines that indicate to the user why results are retrieved are valued
higher by users and gain user trust. We deduce the query term weights from the
term gating network in the Deep Relevance Matching Model (DRMM) and visualize
them as a doughnut chart. In addition, we train a passage-level ranker with
DRMM that selects the most relevant passage from each document and shows it as
snippet on the result page. Next to the snippet we show a document thumbnail
with this passage highlighted. We evaluate the proposed interface in an online
user study, asking users to judge the explainability and assessability of the
interface. We found that users judge our proposed interface significantly more
explainable and easier to assess than a regular search engine result page.
However, they are not significantly better in selecting the relevant documents
from the top-5. This indicates that the explainability of the search engine
result page leads to a better user experience. Thus, we conclude that the
proposed explainable elements are promising as visualization for search engine
users.
