  Large language models (LLMs), such as ChatGPT and GPT-4, are gaining
wide-spread real world use. Yet, these LLMs are closed source, and little is
known about their performance in real-world use cases. In this paper, we apply
and evaluate the combination of ChatGPT and GPT-4 for the real-world task of
mining insights from a text corpus in order to identify research challenges in
the field of HCI. We extract 4,392 research challenges in over 100 topics from
the 2023~CHI conference proceedings and visualize the research challenges for
interactive exploration. We critically evaluate the LLMs on this practical task
and conclude that the combination of ChatGPT and GPT-4 makes an excellent
cost-efficient means for analyzing a text corpus at scale. Cost-efficiency is
key for flexibly prototyping research ideas and analyzing text corpora from
different perspectives, with implications for applying LLMs for mining insights
in academia and practice.
