  This study examines the decoy effect's underexplored influence on user search
interactions and methods for measuring information retrieval (IR) systems'
vulnerability to this effect. It explores how decoy results alter users'
interactions on search engine result pages, focusing on metrics like
click-through likelihood, browsing time, and perceived document usefulness. By
analyzing user interaction logs from multiple datasets, the study demonstrates
that decoy results significantly affect users' behavior and perceptions.
Furthermore, it investigates how different levels of task difficulty and user
knowledge modify the decoy effect's impact, finding that easier tasks and lower
knowledge levels lead to higher engagement with target documents. In terms of
IR system evaluation, the study introduces the DEJA-VU metric to assess
systems' susceptibility to the decoy effect, testing it on specific retrieval
tasks. The results show differences in systems' effectiveness and
vulnerability, contributing to our understanding of cognitive biases in search
behavior and suggesting pathways for creating more balanced and bias-aware IR
evaluations.
