  This registered report introduces the largest, and for the first time,
reproducible experimental survey on biomedical sentence similarity with the
following aims: (1) to elucidate the state of the art of the problem; (2) to
solve some reproducibility problems preventing the evaluation of most of
current methods; (3) to evaluate several unexplored sentence similarity
methods; (4) to evaluate an unexplored benchmark, called
Corpus-Transcriptional-Regulation; (5) to carry out a study on the impact of
the pre-processing stages and Named Entity Recognition (NER) tools on the
performance of the sentence similarity methods; and finally, (6) to bridge the
lack of reproducibility resources for methods and experiments in this line of
research. Our experimental survey is based on a single software platform that
is provided with a detailed reproducibility protocol and dataset as
supplementary material to allow the exact replication of all our experiments.
In addition, we introduce a new aggregated string-based sentence similarity
method, called LiBlock, together with eight variants of current ontology-based
methods and a new pre-trained word embedding model trained on the full-text
articles in the PMC-BioC corpus. Our experiments show that our novel
string-based measure sets the new state of the art on the sentence similarity
task in the biomedical domain and significantly outperforms all the methods
evaluated herein, except one ontology-based method. Likewise, our experiments
confirm that the pre-processing stages, and the choice of the NER tool, have a
significant impact on the performance of the sentence similarity methods. We
also detail some drawbacks and limitations of current methods, and warn on the
need of refining the current benchmarks. Finally, a noticeable finding is that
our new string-based method significantly outperforms all state-of-the-art
Machine Learning models evaluated herein.
