  Airbnb is a two-sided marketplace, bringing together hosts who own listings
for rent, with prospective guests from around the globe. Applying neural
network-based learning to rank techniques has led to significant improvements
in matching guests with hosts. These improvements in ranking were driven by a
core strategy: order the listings by their estimated booking probabilities,
then iterate on techniques to make these booking probability estimates more and
more accurate. Embedded implicitly in this strategy was an assumption that the
booking probability of a listing could be determined independently of other
listings in search results. In this paper we discuss how this assumption,
pervasive throughout the commonly-used learning to rank frameworks, is false.
We provide a theoretical foundation correcting this assumption, followed by
efficient neural network architectures based on the theory. Explicitly
accounting for possible similarities between listings, and reducing them to
diversify the search results generated strong positive impact. We discuss these
metric wins as part of the online A/B tests of the theory. Our method provides
a practical way to diversify search results for large-scale production ranking
systems.
