  Systematic comparison of methods for relation extraction (RE) is difficult
because many experiments in the field are not described precisely enough to be
completely reproducible and many papers fail to report ablation studies that
would highlight the relative contributions of their various combined
techniques. In this work, we build a unifying framework for RE, applying this
on three highly used datasets (from the general, biomedical and clinical
domains) with the ability to be extendable to new datasets. By performing a
systematic exploration of modeling, pre-processing and training methodologies,
we find that choices of pre-processing are a large contributor performance and
that omission of such information can further hinder fair comparison. Other
insights from our exploration allow us to provide recommendations for future
research in this area.
