  Reproducibility is a major concern across scientific fields. Human-Computer
Interaction (HCI), in particular, is subject to diverse reproducibility
challenges due to the wide range of research methodologies employed. In this
article, we explore how the increasing adoption of Large Language Models (LLMs)
across all user experience (UX) design and research activities impacts
reproducibility in HCI. In particular, we review upcoming reproducibility
challenges through the lenses of analogies from past to future (mis)practices
like p-hacking and prompt-hacking, general bias, support in data analysis,
documentation and education requirements, and possible pressure on the
community. We discuss the risks and chances for each of these lenses with the
expectation that a more comprehensive discussion will help shape best practices
and contribute to valid and reproducible practices around using LLMs in HCI
research.
