  There is an increasing demand for algorithms to explain their outcomes. So
far, there is no method that explains the rankings produced by a ranking
algorithm. To address this gap we propose LISTEN, a LISTwise ExplaiNer, to
explain rankings produced by a ranking algorithm. To efficiently use LISTEN in
production, we train a neural network to learn the underlying explanation space
created by LISTEN; we call this model Q-LISTEN. We show that LISTEN produces
faithful explanations and that Q-LISTEN is able to learn these explanations.
Moreover, we show that LISTEN is safe to use in a real world environment: users
of a news recommendation system do not behave significantly differently when
they are exposed to explanations generated by LISTEN instead of manually
generated explanations.
