  Few-shot transfer often shows substantial gain over zero-shot
transfer~\cite{lauscher2020zero}, which is a practically useful trade-off
between fully supervised and unsupervised learning approaches for multilingual
pretrained model-based systems. This paper explores various strategies for
selecting data for annotation that can result in a better few-shot transfer.
The proposed approaches rely on multiple measures such as data entropy using
$n$-gram language model, predictive entropy, and gradient embedding. We propose
a loss embedding method for sequence labeling tasks, which induces diversity
and uncertainty sampling similar to gradient embedding. The proposed data
selection strategies are evaluated and compared for POS tagging, NER, and NLI
tasks for up to 20 languages. Our experiments show that the gradient and loss
embedding-based strategies consistently outperform random data selection
baselines, with gains varying with the initial performance of the zero-shot
transfer. Furthermore, the proposed method shows similar trends in improvement
even when the model is fine-tuned using a lower proportion of the original
task-specific labeled training data for zero-shot transfer.
