  Identifying when to give treatments to patients and how to select among
multiple treatments over time are important medical problems with a few
existing solutions. In this paper, we introduce the Counterfactual Recurrent
Network (CRN), a novel sequence-to-sequence model that leverages the
increasingly available patient observational data to estimate treatment effects
over time and answer such medical questions. To handle the bias from
time-varying confounders, covariates affecting the treatment assignment policy
in the observational data, CRN uses domain adversarial training to build
balancing representations of the patient history. At each timestep, CRN
constructs a treatment invariant representation which removes the association
between patient history and treatment assignments and thus can be reliably used
for making counterfactual predictions. On a simulated model of tumour growth,
with varying degree of time-dependent confounding, we show how our model
achieves lower error in estimating counterfactuals and in choosing the correct
treatment and timing of treatment than current state-of-the-art methods.
