  The emergence of Large Language Models (LLMs) has revealed a growing need for
human-AI collaboration, especially in creative decision-making scenarios where
trust and reliance are paramount. Through human studies and model evaluations
on the open-ended News Headline Generation task from the LaMP benchmark, we
analyze how the framing and presence of explanations affect user trust and
model performance. Overall, we provide evidence that adding an explanation in
the model response to justify its reasoning significantly increases
self-reported user trust in the model when the user has the opportunity to
compare various responses. Position and faithfulness of these explanations are
also important factors. However, these gains disappear when users are shown
responses independently, suggesting that humans trust all model responses,
including deceptive ones, equitably when they are shown in isolation. Our
findings urge future research to delve deeper into the nuanced evaluation of
trust in human-machine teaming systems.
