  This paper explores the effectiveness of using large language models (LLMs)
for personalized movie recommendations from users' perspectives in an online
field experiment. Our study involves a combination of between-subject prompt
and historic consumption assessments, along with within-subject recommendation
scenario evaluations. By examining conversation and survey response data from
160 active users, we find that LLMs offer strong recommendation explainability
but lack overall personalization, diversity, and user trust. Our results also
indicate that different personalized prompting techniques do not significantly
affect user-perceived recommendation quality, but the number of movies a user
has watched plays a more significant role. Furthermore, LLMs show a greater
ability to recommend lesser-known or niche movies. Through qualitative
analysis, we identify key conversational patterns linked to positive and
negative user interaction experiences and conclude that providing personal
context and examples is crucial for obtaining high-quality recommendations from
LLMs.
