  We present a new scientific document similarity model based on matching
fine-grained aspects of texts. To train our model, we exploit a
naturally-occurring source of supervision: sentences in the full-text of papers
that cite multiple papers together (co-citations). Such co-citations not only
reflect close paper relatedness, but also provide textual descriptions of how
the co-cited papers are related. This novel form of textual supervision is used
for learning to match aspects across papers. We develop multi-vector
representations where vectors correspond to sentence-level aspects of
documents, and present two methods for aspect matching: (1) A fast method that
only matches single aspects, and (2) a method that makes sparse multiple
matches with an Optimal Transport mechanism that computes an Earth Mover's
Distance between aspects. Our approach improves performance on document
similarity tasks in four datasets. Further, our fast single-match method
achieves competitive results, paving the way for applying fine-grained
similarity to large scientific corpora. Code, data, and models available at:
https://github.com/allenai/aspire
