  Sepsis is a potentially life threatening inflammatory response to infection
or severe tissue damage. It has a highly variable clinical course, requiring
constant monitoring of the patient's state to guide the management of
intravenous fluids and vasopressors, among other interventions. Despite decades
of research, there's still debate among experts on optimal treatment. Here, we
combine for the first time, distributional deep reinforcement learning with
mechanistic physiological models to find personalized sepsis treatment
strategies. Our method handles partial observability by leveraging known
cardiovascular physiology, introducing a novel physiology-driven recurrent
autoencoder, and quantifies the uncertainty of its own results. Moreover, we
introduce a framework for uncertainty aware decision support with humans in the
loop. We show that our method learns physiologically explainable, robust
policies that are consistent with clinical knowledge. Further our method
consistently identifies high risk states that lead to death, which could
potentially benefit from more frequent vasopressor administration, providing
valuable guidance for future research
