  Artificial intelligence (AI) systems are increasingly used for providing
advice to facilitate human decision making in a wide range of domains, such as
healthcare, criminal justice, and finance. Motivated by limitations of the
current practice where algorithmic advice is provided to human users as a
constant element in the decision-making pipeline, in this paper we raise the
question of when should algorithms provide advice? We propose a novel design of
AI systems in which the algorithm interacts with the human user in a two-sided
manner and aims to provide advice only when it is likely to be beneficial for
the user in making their decision. The results of a large-scale experiment show
that our advising approach manages to provide advice at times of need and to
significantly improve human decision making compared to fixed, non-interactive,
advising approaches. This approach has additional advantages in facilitating
human learning, preserving complementary strengths of human decision makers,
and leading to more positive responsiveness to the advice.
