  Workers spend a significant amount of time learning how to make good
decisions. Evaluating the efficacy of a given decision, however, can be
complicated -- e.g., decision outcomes are often long-term and relate to the
original decision in complex ways. Surprisingly, even though learning good
decision-making strategies is difficult, they can often be expressed in simple
and concise forms. Focusing on sequential decision-making, we design a novel
machine learning algorithm that is capable of extracting "best practices" from
trace data and conveying its insights to humans in the form of interpretable
"tips". Our algorithm selects the tip that best bridges the gap between the
actions taken by human workers and those taken by the optimal policy in a way
that accounts for which actions are consequential for achieving higher
performance. We evaluate our approach through a series of randomized controlled
experiments where participants manage a virtual kitchen. Our experiments show
that the tips generated by our algorithm can significantly improve human
performance relative to intuitive baselines. In addition, we discuss a number
of empirical insights that can help inform the design of algorithms intended
for human-AI interfaces. For instance, we find evidence that participants do
not simply blindly follow our tips; instead, they combine them with their own
experience to discover additional strategies for improving performance.
