  This paper presents the first deep reinforcement learning (DRL) framework to
estimate the optimal Dynamic Treatment Regimes from observational medical data.
This framework is more flexible and adaptive for high dimensional action and
state spaces than existing reinforcement learning methods to model real-life
complexity in heterogeneous disease progression and treatment choices, with the
goal of providing doctor and patients the data-driven personalized decision
recommendations. The proposed DRL framework comprises (i) a supervised learning
step to predict the most possible expert actions, and (ii) a deep reinforcement
learning step to estimate the long-term value function of Dynamic Treatment
Regimes. Both steps depend on deep neural networks.
  As a key motivational example, we have implemented the proposed framework on
a data set from the Center for International Bone Marrow Transplant Research
(CIBMTR) registry database, focusing on the sequence of prevention and
treatments for acute and chronic graft versus host disease after
transplantation. In the experimental results, we have demonstrated promising
accuracy in predicting human experts' decisions, as well as the high expected
reward function in the DRL-based dynamic treatment regimes.
