  For the past few years, we used Apache Lucene as recommendation frame-work in
our scholarly-literature recommender system of the reference-management
software Docear. In this paper, we share three lessons learned from our work
with Lucene. First, recommendations with relevance scores below 0.025 tend to
have significantly lower click-through rates than recommendations with
relevance scores above 0.025. Second, by picking ten recommendations randomly
from Lucene's top50 search results, click-through rate decreased by 15%,
compared to recommending the top10 results. Third, the number of returned
search results tend to predict how high click-through rates will be: when
Lucene returns less than 1,000 search results, click-through rates tend to be
around half as high as if 1,000+ results are returned.
