  A key challenge in sequential decision making is optimizing systems safely
under partial information. While much of the literature has focused on the
cases of either partially known states or partially known dynamics, it is
further exacerbated in cases where both states and dynamics are partially
known. Computing heparin doses for patients fits this paradigm since the
concentration of heparin in the patient cannot be measured directly and the
rates at which patients metabolize heparin vary greatly between individuals.
While many proposed solutions are model free, they require complex models and
have difficulty ensuring safety. However, if some of the structure of the
dynamics is known, a model based approach can be leveraged to provide safe
policies. In this paper we propose such a framework to address the challenge of
optimizing personalized heparin doses. We use a predictive model parameterized
individually by patient to predict future therapeutic effects. We then leverage
this model using a scenario generation based approach that is capable of
ensuring patient safety. We validate our models with numerical experiments by
comparing the predictive capabilities of our model against existing machine
learning techniques and demonstrating how our dosing algorithm can treat
patients in a simulated ICU environment.
