  Transformer-based language models have achieved remarkable success in
few-shot in-context learning and drawn a lot of research interest. However,
these models' performance greatly depends on the choice of the example prompts
and also has high variability depending on how samples are chosen. In this
paper, we conduct a comprehensive study of retrieving semantically similar
few-shot samples and using them as the context, as it helps the model decide
the correct label without any gradient update in the multilingual and
cross-lingual settings. We evaluate the proposed method on five natural
language understanding datasets related to intent detection, question
classification, sentiment analysis, and topic classification. The proposed
method consistently outperforms random sampling in monolingual and
cross-lingual tasks in non-English languages.
