  In AI-assisted decision-making, humans often passively review AI's suggestion
and decide whether to accept or reject it as a whole. In such a paradigm,
humans are found to rarely trigger analytical thinking and face difficulties in
communicating the nuances of conflicting opinions to the AI when disagreements
occur. To tackle this challenge, we propose Human-AI Deliberation, a novel
framework to promote human reflection and discussion on conflicting human-AI
opinions in decision-making. Based on theories in human deliberation, this
framework engages humans and AI in dimension-level opinion elicitation,
deliberative discussion, and decision updates. To empower AI with deliberative
capabilities, we designed Deliberative AI, which leverages large language
models (LLMs) as a bridge between humans and domain-specific models to enable
flexible conversational interactions and faithful information provision. An
exploratory evaluation on a graduate admissions task shows that Deliberative AI
outperforms conventional explainable AI (XAI) assistants in improving humans'
appropriate reliance and task performance. Based on a mixed-methods analysis of
participant behavior, perception, user experience, and open-ended feedback, we
draw implications for future AI-assisted decision tool design.
