  As artificial intelligence (AI) systems play an increasingly prominent role
in human decision-making, challenges surface in the realm of human-AI
interactions. One challenge arises from the suboptimal AI policies due to the
inadequate consideration of humans disregarding AI recommendations, as well as
the need for AI to provide advice selectively when it is most pertinent. This
paper presents a sequential decision-making model that (i) takes into account
the human's adherence level (the probability that the human follows/rejects
machine advice) and (ii) incorporates a defer option so that the machine can
temporarily refrain from making advice. We provide learning algorithms that
learn the optimal advice policy and make advice only at critical time stamps.
Compared to problem-agnostic reinforcement learning algorithms, our specialized
learning algorithms not only enjoy better theoretical convergence properties
but also show strong empirical performance.
