  BACKGROUND: In this study, we investigated the efficacy of current
state-of-the-art neural sentence embedding models for semantic similarity
estimation of sentences from biomedical literature. We trained different neural
embedding models on 1.7 million articles from the PubMed Open Access dataset,
and evaluated them based on a biomedical benchmark set containing 100 sentence
pairs annotated by human experts and a smaller contradiction subset derived
from the original benchmark set.
  RESULTS: With a Pearson correlation of 0.819, our best unsupervised model
based on the Paragraph Vector Distributed Memory algorithm outperforms previous
state-of-the-art results achieved on the BIOSSES biomedical benchmark set.
Moreover, our proposed supervised model that combines different string-based
similarity metrics with a neural embedding model surpasses previous
ontology-dependent supervised state-of-the-art approaches in terms of Pearson's
r (r=0.871) on the biomedical benchmark set. In contrast to the promising
results for the original benchmark, we found our best models' performance on
the smaller contradiction subset to be poor.
  CONCLUSIONS: In this study we highlighted the value of neural network-based
models for semantic similarity estimation in the biomedical domain by showing
that they can keep up with and even surpass previous state-of-the-art
approaches for semantic similarity estimation that depend on the availability
of laboriously curated ontologies when evaluated on a biomedical benchmark set.
Capturing contradictions and negations in biomedical sentences, however,
emerged as an essential area for further work.
