  Large health care data repositories such as electronic health records (EHR)
open new opportunities to derive individualized treatment strategies for
complicated diseases such as sepsis. In this paper, we consider the problem of
estimating sequential treatment rules tailored to a patient's individual
characteristics, often referred to as dynamic treatment regimes (DTRs). Our
main objective is to find the optimal DTR that maximizes a discontinuous value
function through direct maximization of Fisher consistent surrogate loss
functions. In this regard, we demonstrate that a large class of concave
surrogates fails to be Fisher consistent -- a behavior that differs from the
classical binary classification problems. We further characterize a non-concave
family of Fisher consistent smooth surrogate functions, which is amenable to
gradient-descent type optimization algorithms. Compared to the existing direct
search approach under the support vector machine framework (Zhao et al., 2015),
our proposed DTR estimation via surrogate loss optimization (DTRESLO) method is
more computationally scalable to large sample sizes and allows for broader
functional classes for treatment policies. We establish theoretical properties
for our proposed DTR estimator and obtain a sharp upper bound on the regret
corresponding to our DTRESLO method. The finite sample performance of our
proposed estimator is evaluated through extensive simulations. Finally, we
illustrate the working principles and benefits of our method for estimating an
optimal DTR for treating sepsis using EHR data from sepsis patients admitted to
intensive care units.
