  Recent instruction-finetuned large language models (LMs) have achieved
notable performances in various tasks, such as question-answering (QA).
However, despite their ability to memorize a vast amount of general knowledge
across diverse tasks, they might be suboptimal on specific tasks due to their
limited capacity to transfer and adapt knowledge to target tasks. Moreover,
further finetuning LMs with labeled datasets is often infeasible due to their
absence, but it is also questionable if we can transfer smaller LMs having
limited knowledge only with unlabeled test data. In this work, we show and
investigate the capabilities of smaller self-adaptive LMs, only with unlabeled
test data. In particular, we first stochastically generate multiple answers,
and then ensemble them while filtering out low-quality samples to mitigate
noise from inaccurate labels. Our proposed self-adaption strategy demonstrates
significant performance improvements on benchmark QA datasets with higher
robustness across diverse prompts, enabling LMs to stay stable. Code is
available at: https://github.com/starsuzi/T-SAS.
