  The recent advancements in large language models (LLMs) have brought
significant progress in solving NLP tasks. Notably, in-context learning (ICL)
is the key enabling mechanism for LLMs to understand specific tasks and
grasping nuances. In this paper, we propose a simple yet effective method to
contextualize a task toward a specific LLM, by (1) observing how a given LLM
describes (all or a part of) target datasets, i.e., open-ended zero-shot
inference, and (2) aggregating the open-ended inference results by the LLM, and
(3) finally incorporate the aggregated meta-information for the actual task. We
show the effectiveness of this approach in text clustering tasks, and also
highlight the importance of the contextualization through examples of the above
procedure.
