  Transformers represent the state-of-the-art in Natural Language Processing
(NLP) in recent years, proving effective even in tasks done in low-resource
languages. While pretrained transformers for these languages can be made, it is
challenging to measure their true performance and capacity due to the lack of
hard benchmark datasets, as well as the difficulty and cost of producing them.
In this paper, we present three contributions: First, we propose a methodology
for automatically producing Natural Language Inference (NLI) benchmark datasets
for low-resource languages using published news articles. Through this, we
create and release NewsPH-NLI, the first sentence entailment benchmark dataset
in the low-resource Filipino language. Second, we produce new pretrained
transformers based on the ELECTRA technique to further alleviate the resource
scarcity in Filipino, benchmarking them on our dataset against other
commonly-used transfer learning techniques. Lastly, we perform analyses on
transfer learning techniques to shed light on their true performance when
operating in low-data domains through the use of degradation tests.
