  Large language models (LLMs) with chat-based capabilities, such as ChatGPT,
are widely used in various workflows. However, due to a limited understanding
of these large-scale models, users struggle to use this technology and
experience different kinds of dissatisfaction. Researchers have introduced
several methods, such as prompt engineering, to improve model responses.
However, they focus on enhancing the model's performance in specific tasks, and
little has been investigated on how to deal with the user dissatisfaction
resulting from the model's responses. Therefore, with ChatGPT as the case
study, we examine users' dissatisfaction along with their strategies to address
the dissatisfaction. After organizing users' dissatisfaction with LLM into
seven categories based on a literature review, we collected 511 instances of
dissatisfactory ChatGPT responses from 107 users and their detailed
recollections of dissatisfactory experiences, which we released as a publicly
accessible dataset. Our analysis reveals that users most frequently experience
dissatisfaction when ChatGPT fails to grasp their intentions, while they rate
the severity of dissatisfaction related to accuracy the highest. We also
identified four tactics users employ to address their dissatisfaction and their
effectiveness. We found that users often do not use any tactics to address
their dissatisfaction, and even when using tactics, 72% of dissatisfaction
remained unresolved. Moreover, we found that users with low knowledge of LLMs
tend to face more dissatisfaction on accuracy while they often put minimal
effort in addressing dissatisfaction. Based on these findings, we propose
design implications for minimizing user dissatisfaction and enhancing the
usability of chat-based LLM.
