  Citation recommendation is intended to assist researchers in the process of
searching for relevant papers to cite by recommending appropriate citations for
a given input text. Existing test collections for this task are noisy and
unreliable since they are built automatically from parsed PDF papers. In this
paper, we present our ongoing effort at creating a publicly available, manually
annotated test collection for citation recommendation. We also conduct a series
of experiments to evaluate the effectiveness of content-based baseline models
on the test collection, providing results for future work to improve upon. Our
test collection and code to replicate experiments are available at
https://github.com/boudinfl/acm-cr
