  We consider the task of detecting sentences that express causality, as a step
towards mining causal relations from texts. To bypass the scarcity of causal
instances in relation extraction datasets, we exploit transfer learning, namely
ELMO and BERT, using a bidirectional GRU with self-attention (BIGRUATT) as a
baseline. We experiment with both generic public relation extraction datasets
and a new biomedical causal sentence detection dataset, a subset of which we
make publicly available. We find that transfer learning helps only in very
small datasets. With larger datasets, BIGRUATT reaches a performance plateau,
then larger datasets and transfer learning do not help.
