  Many public policies and medical interventions involve dynamics in their
treatment assignments, where treatments are sequentially assigned to the same
individuals across multiple stages, and the effect of treatment at each stage
is usually heterogeneous with respect to the history of prior treatments and
associated characteristics. We study statistical learning of optimal dynamic
treatment regimes (DTRs) that guide the optimal treatment assignment for each
individual at each stage based on the individual's history. We propose a
step-wise doubly-robust approach to learn the optimal DTR using observational
data under the assumption of sequential ignorability. The approach solves the
sequential treatment assignment problem through backward induction, where, at
each step, we combine estimators of propensity scores and action-value
functions (Q-functions) to construct augmented inverse probability weighting
estimators of values of policies for each stage. The approach consistently
estimates the optimal DTR if either a propensity score or Q-function for each
stage is consistently estimated. Furthermore, the resulting DTR can achieve the
optimal convergence rate $n^{-1/2}$ of regret under mild conditions on the
convergence rate for estimators of the nuisance parameters.
