  While transformer-based finetuning techniques have proven effective in tasks
that involve low-resource, low-data environments, a lack of properly
established baselines and benchmark datasets make it hard to compare different
approaches that are aimed at tackling the low-resource setting. In this work,
we provide three contributions. First, we introduce two previously unreleased
datasets as benchmark datasets for text classification and low-resource
multilabel text classification for the low-resource language Filipino. Second,
we pretrain better BERT and DistilBERT models for use within the Filipino
setting. Third, we introduce a simple degradation test that benchmarks a
model's resistance to performance degradation as the number of training samples
are reduced. We analyze our pretrained model's degradation speeds and look
towards the use of this method for comparing models aimed at operating within
the low-resource setting. We release all our models and datasets for the
research community to use.
