  In several medical decision-making problems, such as antibiotic prescription,
laboratory testing can provide precise indications for how a patient will
respond to different treatment options. This enables us to "fully observe" all
potential treatment outcomes, but while present in historical data, these
results are infeasible to produce in real-time at the point of the initial
treatment decision. Moreover, treatment policies in these settings often need
to trade off between multiple competing objectives, such as effectiveness of
treatment and harmful side effects. We present, compare, and evaluate three
approaches for learning individualized treatment policies in this setting:
First, we consider two indirect approaches, which use predictive models of
treatment response to construct policies optimal for different trade-offs
between objectives. Second, we consider a direct approach that constructs such
a set of policies without intermediate models of outcomes. Using a medical
dataset of Urinary Tract Infection (UTI) patients, we show that all approaches
learn policies that achieve strictly better performance on all outcomes than
clinicians, while also trading off between different objectives. We demonstrate
additional benefits of the direct approach, including flexibly incorporating
other goals such as deferral to physicians on simple cases.
